Levels of impact and evaluation
Communication is a two-sided, multistage process which takes effect in multiple dimensions and as such can fail or succeed in different ways.
Target-setting and evaluation as part of communication management
The targets set for communication management planning must therefore be multidimensional. A (PR) evaluation process – during or after the subsequent implementation of individual measures, e.g. media relations, online and event communications – is required in order to determine whether those targets have been achieved. Formative, in-process control looks at whether assumptions, plans and goals still apply, while summative retrospective control looks at whether the intended impacts have taken effect.
Controlling the communication process
To evaluate target achievement and success, it is necessary to analyze the individual phases of communication processes from initiation right through to impact (if any). PR evaluation research has proposed a number of pertinent concepts over the past number of years, operationalized in a range of effect levels termed Output, Outtake, Outgrowth, Outcome and Outflow, but the definitions applied have tended to vary in some cases. Against this background, theorists, communication managers and controllers in the German-speaking world have conducted an extensive discussion process culminating in consensus on standard points of reference – the DPRG/ICV Framework for Communication Controlling (Rolke/Zerfaß 2010). The concept was adopted in spring 2009 by the German Public Relations Association (DPRG) and the International Controlling Association (ICV) as industry standard, and has already been accepted by leading companies and other professional organizations such as the Kommunikationsverband (Communication Association).
DPRG/ICV framework for communication controlling - outline [Download PDF]
The DPRG/ICV framework is not a "fully calculable and predictable" model. Rather, it is a discussion tool that seeks to provide orientation on a meta level and provide a workable context for the plethora of existing evaluation methods and parameters. The effect levels reference framework is a complex input-output scheme with a five-stage effect side. The five stages are:
- Input (the investment/effort that is put into communication). The resources invested embrace human and financial capital expended on communication. Both these kinds of capital can be measured in cost categories.
- Internal output (performance of the actual organization). This is about process efficiency, which can be demonstrated in terms of things like budget compliance, throughput times and error rates, and it is also about the quality of the activities of communication departments and agencies. An important benchmark here is the satisfaction of (internal) clients. The focus is still on the organization itself and the initiation of actual communication processes.
- External output (contact offerings that are created). This effect level refers to the range and content of communication offerings that are accessible to reference groups/recipients. Characteristics of media outlets and communication channels are investigated on the basis of parameters such as media report clippings, website visits, and share of voice as the proportion of corporate reports versus total media reporting in an industry. These are necessary criteria for the success of communication processes, but do not yet constitute effective indicators of successful achievement of communicative objectives vis-à-vis reference groups.
- Direct outcome (effectiveness in terms of raising perception levels and knowledge). Perception, utilization and knowledge are changes effected at the stakeholders' end. Awareness, time spent on site, readers per issue, recall and recognition are typical parameters used to demonstrate information generation in communicative interactions.
- Indirect outcome (extent of impact on opinions/intentions). This stage relates to impact understood as the actual goal of all communication processes. Opinions, attitudes, emotions, behavioral disposition and behavior/action of stakeholders can be determined on the basis of indicators such as brand image, reputation indices, employee commitment, buying intention etc.
- Outflow (value-adding objectives enhanced by communication). Communication processes can take effect by impacting on strategic and/or financial objectives in the value-adding process, and by affecting tangible and/or in-tangible capital-building resources. Outcome measures here include sales, project completions, cost reduction, and reputation/brand values. Unlike the standardized evaluable effect levels mentioned thus far, communication's contribution to overarching corporate strategy must be evaluated on a company-specific basis, as each company is positioned differently on the market and in society. A number of general conclusions can be drawn nonetheless. One approach is to proceed from the company's generic strategy map, which identifies typical influencing factors for individual areas of action and which can be enhanced through the use of communicative value links (value drivers) and parameters. Company-specific implementation is possible through the use of flexible control systems such as scorecards. Standardized methods to evaluate brands and image/repuation are used in addition to assess the intangible capital created primarily through communication. Outcome measures here include sales, project completions, cost reduction, and reputation/branding values.
Empirical findings
The various levels of the communication process are evaluated in very different ways in the real-life corporate world. A survey among 1,955 senior communication managers in Europe (Zerfass et al. 2010) showed that the main priority at present is very definitely the company's mass media impact and the range of the company's own communication platforms on the internet and intranet (external output). 76 percent of respondents say they try to create more transparency in this area. As many as 54 percent say they evaluate their effectiveness in getting messages across to communication partners (direct outcome). Only 42 percent generate statistics on their own efficiency (internal output); the exertion of influence as the true goal of communication is likewise accorded only minor importance (42 percent). Of crucial significance, and shocking from the point of view of value-oriented communication management, is the fact that the minorityof respondents say they evaluate expenditure of resources (input; 38 percent) and impact on corporate goals, including creation of intangible assets (outflow; 25 percent).
Refinement and modifications
The reference framework outlined here can be refined for use in the real-life business world. It makes sense in particular to distinguish between direct and indirect communication measures. While a company's messages in terms of direct communication (e.g. through corporate events, corporate publishing, corporate websites) create immediate contact opportunities (output), indirect communication (e.g. media relations, word of mouth, viral communication on the web) is directed initially at gatekeepers, opinion leaders and multipliers. Hence, availability and impacts (output/outcome at the mediator level) need to be achieved and measured there first of all before the message – via actions induced in this manner, e.g. reporting in a journal or weblog – even reaches the stakeholders who are actually being targeted.
References
Kaplan, Robert S./Norton, David P. (2004): Strategy Maps. Der Weg von immateriellen Werten zum materiellen Erfolg. Stuttgart: Schäffer-Poeschel.
Lindenmann, Walter (2003): Guidelines for Measuring the Effectiveness of PR Programs and Activities. Gainesville (FL): IPR. (PDF)
Rolke, Lothar/Zerfaß, Ansgar (2010): Wirkungsdimensionen der Kommunikation: Ressourceneinsatz und Wertschöpfung im DPRG/ICV-Bezugsrahmen. In: Pfannenberg, Jörg/Zerfaß, Ansgar (Eds.): Wertschöpfung durch Kommunikation. Kommunikations-Controlling in der Unternehmenspraxis. Frankfurt/Main: Frankfurter Allgemeine Buch, pp 50-61.
Paine, Katie D. (2007): Measuring Public Relationships. The Data-Driven Communicator's Guide to Success. Berlin (NH): KDPaine & Partners.
Watson, Tom/Noble, Paul (2007): Evaluating Public Relations. 2. Auflage. London: Kogan Page.
Zerfaß, Ansgar (2009): Immaterielle Werte und Unternehmenskommunikation – Herausforderungen für das Kommunikationsmanagement. In: Möller, Klaus/Piwinger, Manfred/Zerfaß, Ansgar (Eds.): Immaterielle Vermögenswerte: Bewertung, Berichterstattung und Kommunikation. Stuttgart: Schäffer-Poeschel, pp 23-47.
Zerfaß, Ansgar (2007): Unternehmenskommunikation und Kommunikationsmanagement: Grundlagen, Wertschöpfung, Reputation. In: Piwinger, Manfred/Zerfaß, Ansgar (Eds.): Handbuch Unternehmenskommunikation. Wiesbaden: Gabler, pp 21-70.
Zerfass, Ansgar/Tench, Ralph/Verhoeven, Piet/Verčič, Dejan & Moreno, Angeles (2010): European Communication Monitor 2010. Status Quo and Challenges for Public Relations in Europe. Results of an Empirical Survey in 46 Countries (Chart Version). Brussels: EACD, EUPRERA. [www.communicationmonitor.eu]
