Interview: Communication measurement is a challenge all over Europe
In the first issue of the international journal “Corporate Communications” 2017 Ansgar Zerfass, Dejan Verčič and Sophia Charlotte Volk examine the status quo of communication evaluation and measurement practices in communication departments of companies, non-profits, and other organizations across Europe. In the interview with communicationcontrolling.de the main co-author Sophia Charlotte Volk gives an outline of and reflects on findings and practical implications of the study. Furthermore, she depicts reactions to the presentation of the findings at the International Public Relations Research Conference (IPRRC) in Miami in 2016.
communicationcontrolling.de: What is the study about?
Sophia Charlotte Volk: There have been on-going discussions on how communication supports organizational goals and much has been written about it. However, little is known about how evaluation and measurement activities are actually implemented in practice. Previous studies have revealed that evaluation is often limited to the output level, counting press clippings or visitors at events or websites, and tends to ignore how communication impacts organizational targets. One reason might be the lack of expertise to do so. So our rationale is that evaluation skills are a crucial prerequisite for proving communication success. If practitioners do not know how to measure, how shall they collect valid data? Looking at the skill level is the innovative part of the study. More than 1,600 senior communication professionals working in organizations across Europe were surveyed, asking how advanced their knowledge of evaluation and measurement is, which methods are applied and how organizations benefit from using measurement insights.
cc.de: Why did you choose to present it at the International Public Relations Research Conference (IPRRC) in Miami?
Volk: The IPRRC one of the top venues for presenting new PR research and for interacting with scholars and PR professionals. IPRRC has a unique presentation format with informal roundtable sessions in which ten to twenty participants actively discuss the research presented. This allows for great conversations and the direct diffusion of knowledge between academia and the corporate world.
cc.de: What are key findings regarding the measurement skills of communication professionals?
Volk: Our study reveals that the surveyed practitioners have solid skills for interpreting data or for managing surveys, performing content analysis or running social media analytics. But only an alarming minority of the practitioners reports high competencies when asked about calculating reputation or brand value or constructing communication scorecards. Precisely these skills are however necessary to prove the success of communication activities.
Another finding concerns significant skill level differences across different types of organizations. Our results indicate that the pressure to document communication success appears higher in joint stock companies than in public service or non-governmental organizations, where skill levels are lower. In order to resolve this situation, the evaluation skills of communication professionals need to be advanced through further education and training.
cc.de: On what measurement practices do the organizations focus?
Volk: Throughout Europe, measurement is still heavily based on monitoring outputs, with 83 percent of the organizations documenting media clippings and 69 percent tracking internet use. Only 36 percent of the surveyed organizations focus on evaluating the impact of communication on the organization’s intangible and tangible resources. But without measuring the costs (input) and the impact on organizational goals (outflow), it is impossible to reliably document communication effectiveness and efficiency. Evaluation must address all relevant phases of the communication process.
cc.de: How was the feedback to your paper at the conference?
Volk: Both scholars and practitioners had a strong interest in the topic because there have been perennial discussions about it and the findings support many earlier studies in other world regions.
A wide range of models of PR and communication exist and they all use different terms, including input, output, outtake, outflow, outgrowth, effect, result or impact. An even wider range of metrics and methods for evaluation exists for each stage. The German DPRG/ICV framework is just one among many other models. It categorizes the evaluation stages as input, output, outcome and outflow, which resembles the final stage “impact” or “organizational outcome” in other models. The German framework is very sophisticated in a sense that it depicts the influence of communications along the entire value creation process of the organization, combining management logic, accounting concepts and communication theory.
Overall, the framework has received little attention outside the German-speaking world. However, a few leading experts, Fraser Likely or Tom Watson for instance, have praised the framework as an “encouraging effort”. Viewing the many different models though, there remains a need to develop a standard model and a common language to explain communication effects similarly. Therefore, the international Task Force on Standardization of Multi-Stage PR Measurement Models was created in 2015 to deal with this concern.
cc.de: Did the conference help to generate new ideas?
Volk: The discussions at the conference stressed the importance of on-going training and education in social science research techniques as well as sophisticated valuation methods and management concepts, e.g. scorecards. Should universities and education facilities train future communication staff to apply specific methods? Or should they provide students with a good overview and basic comprehension of the great number of measurement methods, as tools are changing so fast?
The conference showed once again that there remains a lack of standards despite the efforts of industry organizations like the international Association for the Measurement and Evaluation of Communication (AMEC) and the Institute for Public Relations (IPR) Measurement Commission or joint efforts of the PR industry and academia.
cc.de: Why was the paper selected as one of the top three papers?
Volk: Our research demonstrates immediate transference and real-world implications for public relations practitioners and the industry. We argue that the challenge is threefold: first of all, communication professionals have to develop adequate evaluation skills; secondly, they have to focus evaluation on relevant questions and aspects of the communication process; and thirdly, they have to exploit evaluation insights to advance and manage future activities. Our paper points out several steps for improvement and practice: further training, focus on all stages of communication effects, strategic use of measurement data, and setting standards.
cc.de: Does the study offer interesting starting points for future research on communication evaluation and measurement? What are next steps to take?
Volk: It would be interesting to ask what other barriers to evaluation and measurement exist apart from skills, e.g. budget and time constraints, lack of top management support or lack of pressure to document the value contribution. Another interesting topic is to investigate how practitioners explain the value of communication to top managers if they predominantly measure outputs and not outflow. And: What do CEOs expect in return?
The international Task Force on Standardization of Multi-Stage PR Measurement Models addresses some of the mentioned aspects and looks into the use of standard models in practice and the role of associations. For now, we aim to develop a more holistic understanding of creating communication value at the edge of communication and management theory. Scholars have to engage in more theoretical discussions of how communication contributes to revenue generation, e.g. by reputation building, and cost reduction, e.g. by crisis prevention.
cc.de: What findings do you hope to present at the IPRRC in 2021?
Volk: If we repeated the study in 2021, we would like to present data that reveals that the overall skill level has increased over time. Furthermore, we would like to see that evaluation methods document effects along all relevant phases of the communication process. By 2021, measurement and evaluation of communication should be perceived as an on-going strategic process. Data should be strategically integrated into management processes to better steer future activities and big data applications should be utilized for planning purposes.
With regards to the Task Force initiative, it would be great to see a new standardized framework that depicts the entirety of communication effects and uses agreed-upon terminology and metrics.
About Sophia Charlotte Volk
Sophia Charlotte Volk is research associate and PhD candidate at the Department of Strategic Communication at the University of Leipzig as well as project leader in the research program "Value Creating Communication" of the Academic Society for Corporate Management and Communication. Her doctoral dissertation research examines the state of the art of comparative research in communication science, with the objective of developing scientific quality standards for comparative communication research. Further research interests include value creation through communication, strategic communication alignment, and evaluation and measurement. Volk is a member of the international Task Force on Standardization of Multi-Stage PR Measurement Models.
Reference:
Zerfaß, A., Verčič, D., & Volk, S. C. (2017). Communication Evaluation and Measurement: Skills, Practices and Utilization in European Organizations. Corporate Communications – An International Journal, 22 (1), 2-18.
See here for the complete paper.
back to News
