written by Kristin Köhler

The proof of performance

Starting with a Workshop Day on Wednesday, participants have been introduced to measurement principles but also advanced techniques and best practices by experienced practitioners like Katie Paine (KD Paine & Partners) who talked about social media measurement from an international perspective (see also the interview with Katie on communicationcontrolling.com) or Harald Josse (BrandControl) who highlighted the importance of measurement also in a recession. Like his other colleagues from evaluation companies carrying out the workshops he stressed the need for the proof of value in communications. Ludwig Schönefeld from Hoerbiger Holding presented participants how a medium-sized company applied a scorecard and communication metrics in its corporate communications. The workshops provided participants with on-hand data and impressions how to improve measurement of their communications.

Task Force - Value Creation, German Public Relations Association (Dr. Harald Josse, Prof. Dr. Ansgar Zerfaß, Dr. Christopher Stock, Christoph Lautenbach, Ludwig Schönefeld - from left)

Some things don’t change

Sebastian Vesper, editorial director Haymarket Germany, opened the discussions on Thursday, the first plenary day. He focused on new developments and consequences occurring through social media as a revolution type of development. According to his own statement, new media will change the communications strategy of businesses. The question wouldn’ t be if social media is relevant and sustainable but how it is! He passes the question to his three morning session’s sparring partners Björn Edlund (Head of Group Communications, Royal Dutch Shell), Marshall Manson (Director of Digital Strategy, Edelman) and Christoph Keese (President Public Affairs, Axel Springer AG).

Björn Edlund opened the scene with his practical insight from Shell. Being responsible for the Reputation Management internally and externally the question to which extent social media affects the communications strategy is inherent to him. For him, some things don’t change, so is PR managers’ role in an engineers-driven company.  But what will change is the strategic focus of communications itself: community-building, engaging people, relationships and alliances will be the future assets a communication department could provide, especially in a business dealing with the energy challenge significantly as part of its business plan. Adequate evaluation tools for social media are at the centre of Edlund's agenda. Shell has established a "Reputation Tracker" to identify issues being relevant for the business across the whole world. Next to issues management in social media Shell installed a corporate blog that is tracked regarding its usage and reach as well. "Shell Dialogues" has more than 32.000 visitors and 1.500 participants worldwide. The quantitative as well as qualitative auditing is essential for Edlund. A detailed case study will follow on communicationcontrollig.com shortly. His conclusion: Measurement’s important roles are innovations, which help to go beyond perceptions and opinions, especially in the new media landscape to encourage informed behaviour. A return on relationships would be an indicator for good communications in the new media.

 

The New Paradigm

Marshall Manson from Edelman drives the discussion further. He also focuses on the aspect of social media being able to build relationships. Originally, they were formed on the basis of proximity. Technology has given the ability to form relationships in another way. He still sees a tremendous value in traditional media – they does matter! But social media have to be implemented in corporate communications strategy additionally. Furthermore, social media are not rocket science: listen - engage - measure is his simple statement regarding analysis. Social media is arguably the most measurable medium. According to Marshall, you could measure nearly every activity you undertake online, with better reliability than most traditional measurement techniques. However, most businesses aren’t yet sure how to interpret the measurements that they produce. Understanding social media measurements in terms of business value remains the primary challenge: "Shape the metrics to fit the objectives!" There would be no set of metrics being perfect for every circumstance or campaign.

Therefore, Marshall demands a new way of thinking - measurement models that take into account a new suite of criteria: significance of conversation, share of conversation, spreadability, value of interaction, and many more. And in the end, meeting objectives will still be the ultimate and most important determinant of success. Too many of the metrics used would be misleading at best and dishonest at worst. Practitioners should stop misleading themselves and their constituencies, so his overall conclusion.

Christoph Keese from Axel Springer held the last morning's session. In his presentation „Challenging the freebie culture on the web: new ways to monetarize quality journalism“ he has given an outlook on the challenges for paid web content and the future of quality journalism and how media enterprises are forced to change their business model due to social media.

 

The Severstal Story: The important role of research in a transformation process

Sandra Macleod (Group Chief Executive, Echo Research) and Vadim Saveliev (Senior Vice-President, Communications and Public Relations, SeverStal) presented „The Rise and Rise in the Importance of Corporate Reputation as Russia's Largest Steel Producer Repositions Itself on a New International Stage“. In the Severstal story, the company used research to support the transformation process. The whole story about the award-winning project you can read on communicationcontrolling.com.

communicationcontrolling.com also spoke to Sandra about current challenges in the field of evaluation of communications before the Summit. You can read the interview here.

 

About „witch doctors“ and PR evaluation

Frank Ovaitt (President & CEO IPR) let participants in his keynote speech know which four things it only took him five years to learn. To summarize, in his opinion communication professionals are still seen as „witch doctors“ which gain their knowledge through trial and error and find solutions that way over time. But like a sick person also businesses prefer a doctor who understands the methods and techniques of modern medicine. This exactly is what research is doing for PR! Frank talked about IPR’s evaluative research approach and the efforts to establish standards and best practices. He also sees social media in the centre of attention – as it even overtakes porn meanwhile. PR professionals who understand research will rule the field – so his conclusion at the end. communicationcontroling.com has spoken before the Summit with Frank about IPR's current activities and future trends in communication controlling. Here you can read the whole interview.

Frank Ovaitt

„Not everything that counts can be counted, and not everything that can be counted counts“ (Albert Einstein)

Starting with the famous physician, Neil Martinson (Director News & PR, the UK’s Central Office of Information) set the scene for the afternoon's session: In his presentation „Putting the Citizen at the Centre of Communications through Measurement and Evaluation“ he calls evaluation a „da Vinci code” and demands effectiveness and value for money at the same time. And he could give a very good explaination for his demand: The UK COI has benchmarked five evaluation agencies providing them with an identical brief, 138 items of identical media coverage and asked for a media analysis. The outcomes:

• How many people? Range 46m –93m
• Cost per thousand: Range 57p –£2.21
• Favourability/tone: positive Range 17% -100%, neutral range 0% -54%

This is just a brief overview of the results - however quite interesting and arousing. As you can see evaluation doesn’t equal evaluation. Therefore, Neil demands a better form of holistic evaluation and some core mandatory standards.

 

The comparative effectiveness of advertising and PR

David Michaelson (President, Echo Research) and Donald K. Wright (Professor of Public Relations in the College of Communication, Boston University) found out that PR is not superior to advertising universal! In a 2008 study „The Comparative Communications Effectiveness of Advertising and Public Relations“ they researched whether consumers differ in their decision-making process regarding to the media type. By analyzing adult newspaper readers they wanted to know if editorial coverage and advertising perform differently on key measures. They couldn’t find significant differences between ads and stories for either credibility measures, therefore the concept that performance of PR efforts outreach advertising could not be supported by their study design. But a pattern of increasing believability and effectiveness and decreasing neutrality continues with the presence of an editorial; brand preference increased when editorial is in addition to an ad. As a result it can be said, that the two elements work effectively together. Editorial is especially important at the beginning of a marketing campaign, ads for rebuilding awareness. For product PR that all makes sense, but what’s the matter with other campaigns especially aiming not at selling but mobilisation or advocacy? The study has gained high interest by all participants who called for further research.

 

What’s the matter with AVEs?

David B. Rockland (Partner and Managing Director of Ketchum’s Global Research Network and CEO of Stromberg Consulting) talked about  „Why all the fuss about AVEs?“ Some in PR hold the view that editorial space is worth more than paid advertising. Theoretically, PR professionals may weight an AVE value to more accurately reflect a higher value of a PR placement. Widely used methods for weighting figures are PR multipliers, which generally range from 1.5 to 6 or more. But according to David instead of multipliers an algorithm or scoring system for each media mention based on criteria meaningful to the client should be used! Or in other words: Would anyone give a damn dollar for bad coverage? So possibly, even a discount value must be added, not a multiplier. However, David named the following reasons that make the use of AVEs so popular among the measurment community:

• Cost: Relatively less expensive to quantify ROI for a PR program or initiative.
• Clarity: Assigns a dollar value to PR, which may be better understood by management.
• Complementary: Provides additional context when used with other ROI methods or measurements.
• Concision: Data for calculating figures are readily available from vendors, which eliminates time spent on number crunching. This provides faster results.
• Correlations: New research suggests it substantially improves correlations to outcomes.

But there is a list if cons too which shows what AVEs are not able to provide. Especially the last point, the lack of being an adequate key figure for social media measurement, will limit the widespread use in the near future due to new media tools gaining more and more importance in corporate communication processes.

• Cannot capture the outcome of a PR campaign, limiting PR to its placements in the media.
• Cannot measure the variety of messages delivered in the media in relation to the tamed messages in advertisements.
• Cannot measure the value of keeping a client out of the media spotlight.
• Cannot properly distinguish between placements in noteworthy columns or sections of a publication and generic or less desirable areas of a publication.
• Cannot measure social media forms, such as blogs and message boards.

 

 

More practical insights

Roland Kuntze (Head of External Communications, O2 Germany) and Sascha Kynast (Cision Germany) explained the evaluation process in his company. Working with defined key performance indicators he agreed targets with the management board and reports on a quarterly basis about the outcomes. Next to KPIs the analysis includes share of voice, reach, AVEs, favourability ratings, media profiles – all for different product sectors and in a national and international benchmarking. Kuntze explained as well O2’s social media approach. The online media sector is of high importance for the company. O2 is differentiating social media regarding its relevance for the business in A-, B-, C-social media. A-social media are tracked by an evaluation company. Social media are seen as a better indicator for public opinion. On the one side Roland uses it as a tactical tool for tracking topics, risks and issues management. On the other side as an early warning tool to identify critical consumer topics. In four steps the O2 social media workflow could be summarized:

1. Social Media Audit: Finding of the most important and relevant platforms.
2. News Letter and Alert Service: Systematisation and aggregation of user-generated content; Editorial service.
3. Analyzing Social Media: Difference in parameters, stronger focus on products; Threads of discussion; Sentiments: factual or emotional.
4. Semantic modelling and automatisation of content analysis: Step 4 is seen as the future of social media analysis that is not yet practiced.

Michael Hoffman (Head of Corporate Communication, Navigon AG) and Ingrid Moorkens (Ausschnitt Medienbeobachtung) talked about how his company is assessed in social media. Hoffmann described Navigon as a company operating in a very online-related industry. Navigon has a remarkable presence both in traditional as well as in social media as analysis found out. So content analysis is still important, but supplemented by social network analysis. Differences in traditional and social media can be stated in choice of topics and favourability as well as the level of tonality. Social Media is more fragmented at the same time leading to a higher polarisation (more strongly negative or positive tonality posts compared to traditional media). Discussion networks could be identified in which Navigon is very present and talked about. Therefore, these A-social media are in the centre of analysis:

• The main actors are customers, potential customers, affiliate partners.
• Topics are often user-generated: user problems, demands, and speculations.
• Tonalities are polarized, volatile, influenced by speculations.

Navigon is trying to professionalize its presence in social media. The main goals are an overview and keep it in sight, to extend the one-voice policy concept, actor-differentiated communications, the development of an individual voice of Navigon in social media and to develop a set of guidelines for social media communications. The implementation process started with the coordination of the interactions between Marketing, support and corporate communications. Navigon has an own presence in the social media: a Twitter account, an own Facebook site, presence on youtube and GPSbusinessnews. Additionally, relations with social media operators as multipliers are aimed at. The particularities of discussions in social media are considered in corporate communications as well as in product development. Product developers are integrated in active communication (e.g. chat with experts) as well. Social media is seen – beside traditional media - as the media platform for the future, so Michael. This indicates the need to analyse the platforms continuously and to communicate within regularly.

 

Stimulating the international debate

Initiated by AMEC, Benchpoint and renowned academics Prof. Dr. Ansgar Zerfaß (University of Leipzig), Prof. Donald K. Wright (Boston University) and Dr. Tom Watson, Bournemouth University an international benchmark study was initiated before the Summit. The "Global Survey on Communications Measurement 2009" invited PR professionals working in companies/organisations, agencies and evaluation companies to participate. The survey aimed at identifying current trends and developments in the field of evaluation and measurement on an international level. 520 professionals participated and provided information about their expectations of (internal) costumers, the proof of communication's value especially in recession times and much more. Some of the key findings are:

• More people are measuring communications effectiveness than ever before, and more people are personally involved in measurement.
• There are more “somewhat satisfied” than “very satisfied” for external communications tools, which indicates there is much more work to be done to define and refine these tools.
• There are no clear winners when it comes to particular tools. Each one has its adherents.
• Still no clear consensus on measures or methodology. Evidence that people are moving more to measuring outcome rather than output, reversing the pattern in the previous 2004 survey.
• Clippings the number one measurement tool, in terms of use and satisfaction. Opinion polls and benchmarking are less popular than before. Media evaluation tools and dashboards are rising in popularity. Even AVEs have gained a point or two.
• Cost and time remain the major barriers to measurement.
• There is strong belief that ROI on communication can be measured, and there is universal interest in the Holy Grail – a workable ROI measuring tool.
• Opinion remains divided on whether measurement is good, or impractical and expensive – another challenge for the industry.

The results in detail and its implications for communications measurement will be published soon on communicationcontrolling.com

Ansgar Zerfaß could compare the findings with the European Communication Monitor, an annual European research on PR trends: While many consultants and researchers still focus on new evaluation methods, only 19% of senior professionals in Europe think this is relevant. However, every second says “linking business strategy and communication“ is the most relevant issue within the next three years (the number 1 issue in European PR since 2007). These results clearly indicate that a variety of methods is needed, but also a common framework to depict both the communication process and the link to organizational goals. Therefore, Ansgar introduced the joint framework of the German PR Association (DPRG) and International Controller Association (ICV) to the Summit participants as a first step towards institutionalizing a framework that allows to define how CEOs evaluate communications – and not the other way around! The "DPRG/ICV framework for communication controlling" comprises six levels for management of communication activities. In decreasing order of controllability through communications management, the stages of input, output, outcome and outflow designate measuring ranges, exemplary variables and the respective measurand for communications controlling.

[Download PDF]

 

The next generation

The closing summit debate asked for the next generation: What will media evaluation look like in the future? Christopher Storck (Head of Task Force - Value Creation, German Public Relations Association and Principal, HeringSchuppener), asks in the closing remark if PR firms will finally understand that evaluation proves the effectiveness of their activities and work consistently with evaluation companies? Christopher supposes that companies are reluctant to pay much for PR evaluation. There would  be two reasons why PR managers don’t spend more money on media analysis and stakeholder research:

• They don’t see a proper return on this investment.
• They don’t have enough budget for doing their job effectively.

What are the reasons for this? Christopher says research results are often not actionable and budget sizes reflect how much value creation senior managers expect from PR. There is only one solution for him: Corporate communication needs to be integrated in the strategy management system of a company. Media coverage is only one of many contact points between companies and their stakeholders. Analyzing it does not tell anything about how stakeholders perceive a company, what their attitude towards this company is and whether this will make them cooperate with this company or not. If corporate communication wants to increase its impact on business, it needs more and better measurement tools and management skills including all stakeholders – so Christopher's final result (Christopher Storck on communicationcontrolling.com).

Giselle Bodie (Managing Director, Cision UK and Director, AMEC) talked about an evolution, not revolution of future media evaluation. There will be an increased need for PR to demonstrate value especially in crisis or recession times. She does not believe in an "one size fits it all" industry approach, but in the "science behind the art "and in corporate communications’ need to provide evidence for their effectiveness.

Andreas Westermann (Managing Director, AUSSCHNITT Medienbeobachtung) summarized the challenges by social media: chaos communication, dialogue communication, network communication. Media evaluation must be integrated in this process of a changing media landscape and more advanced stakeholders in the online world. For evaluation, new types and methods, stakeholder and network analysis must be integrated. He calls for strategically developed methods and tools for social media measurement being part of the overall communication controlling process.

Joan Sola (CEO, Acceso Group) posed the question whether research and evaluation companies should look at expanding their business to take in reputation management, crisis management, and other new areas which have a research basis? He constitutes a potential conflict between consultancies and evaluation companies. The balanced position for him is interpretation: Research and evaluation should stay independent. Evaluation companies could have an informal consultancy role (brief extraction, results presentation) but reputation management, crisis management and other areas are part of the consultancies’ business.

Plenary session

All in all, there have been three days with a lot of presentations, discussions, controversies and different views about communications measurement in a changing media landscape. To summarize, I would like to end with Sebastian Vesper: “Media is changing, but some things will never change: Good people, continous learning.” Evaluation practices have to adopt the recent and future developments, have to become more advanced to integrate a measuring on an outcome and outflow level to prove communications’ value for organisations. But the good news is: evaluation already is and will become even more important, especially in a crisis or recession PR has to demonstrate its value for the money invested and its nature as an immaterial asset. Also social media will not change this pattern – the new communication tools have to be integrated in the communication strategy. Evaluation could do a lot for this implementation process and adequately use of the new opportunities. It will be seen which new methods and technologies will occur and fit best to social media measurement – but somehow or other communication evaluation is on the forefront seizing its essential role in corporate communications and communications management.

More information you can find on the Summit website:
http://www.ameceuropeansummit.org/summit/

About AMEC

AMEC stands for the international Association for Measurement and Evaluation of Communication. It is the fast growing global trade body for companies in research, measurement and evaluation with 57% of members now outside UK and trend continues. Copies of the full International Business Monitor Report are available to non-members of AMEC priced at £1,500.

About IPR
The Institute for Public Relations is an independent nonprofit foundation dedicated to the science beneath the art of public relations. Located at the University of Florida, the Institute supports public relations research and education to bring together the academic and professional communities.

About DPRG
DPRG is the professional association for all Public Relations experts in Germany (consultants and in-house practitioners). Since 1958 we represent the interests of our members, inform and support them in doing their job. Furthermore, we focus on current topics in the area of Public Relations. Those are, amongst others, ensuring our members adhere to PR codes of conduct, improving scientific standards about the field, developing and establishing educational standards and providing substantial membership services.

deutsch english