Evaluating Communication Campaigns

Health communication and marketing campaigns that promote positive behavior change are a cornerstone of public health and behavioral science. Designing and implementing quality campaigns on a tight budget and in an urgent timeframe is a challenge that most health communication professionals share. Research and evaluation are critical for a successful campaign. CDC is using leading research and evaluation methods to develop quality campaigns, while keeping costs low and sticking to tight timelines.

A great example is CDC’s design and implementation of The Domestic Readiness Initiative on Zika Virus Disease, also known as the Domestic Zika Campaign. This bilingual digital campaign sought to increase Zika awareness among the general population and expectant parents in the United States and its territories. Campaign messages were also developed to change knowledge and attitudes regarding Zika transmission, beliefs about the risks of Zika, and the perceived ability (self-efficacy) to protect oneself against Zika.

CDC identified several methods to get tailored messaging to our target audiences. Based on the work for this campaign, there are 10 methods that can help you reduce costs and improve the efficacy of your communication or health marketing campaign.

Formative Research

1. Use “mixed” methods or alternative approaches. When feasible, mixed assessment techniques can enhance and complement different types of data collection. Your budget may not allow you to conduct in-depth formative research, but you can use one primary research or evaluation method and then supplement with other methods that are less costly. For example, conducting qualitative focus groups might not account for all of your audience segments, but you could perform a literature search to complement your qualitative data. Additionally, when circumstances dictate, sometimes substituting one evaluation activity for another may be just as effective (or close to it). During the Domestic Zika Campaign we used “triad” interviews, which only involved three participants, when we did not have the time or the budget to conduct a full suite of focus groups, and this was supplemented with previous survey data on closely related topics.
2. Invest in market data. Public, non-proprietary market data, such as data from the US Census, PEW, and Gallup, allow you to get to know your audience at no cost. Outside marketing firms may also be willing to share “older” proprietary data that may be from a year or two ago at little or no cost. During the Domestic Zika Campaign, secondary data donated from Annenberg surveys(?) were key to our analysis of each of our target demographics’ media habits and informed the mass media and digital media strategies and plans. The cost is your time in building relationships with like-minded partners and analyzing the data for a specific purpose.
3. Identify alternative data. When traditional surveying of knowledge, attitudes, and behaviors is not feasible, sometimes you can use alternative data (usually less expensive). During the Domestic Zika Campaign, we did not have sufficient time to get approval for our survey from the Office of Management and Budget (OMB) and field a traditional campaign evaluation survey. Instead, to meet our tight timeline, we acquired data about our audiences’ insecticide purchasing behavior in the stores where our advertisements had been placed, and compared them to similar product purchasing data from stores that did not display our advertisements. Building partnerships with organizations who have data, even from related issues, that can offer insights to you is a great long-term investment in both implementing and evaluating programs—especially when we have a shared commitment to protecting people from diseases.

Monitoring & Evaluation

4. Take advantage of social media analytics. Social media platforms like Facebook, Twitter, Instagram, and Google AdWords allow you to target specific geographic areas and have built-in analytics tools that the user has access to at no additional cost. The Domestic Zika Campaign used data analytics to look at the performance of specific advertisements and the corresponding click-through rate to Zika-related campaign websites. Using social media platforms as a primary channel for the campaign allowed for more precise audience targeting, rapid message modifications, and real-time metrics. Using the data, we developed tailored advertisements that engaged key audiences and encouraged them to take preventive actions. For example, one of our treatments, the “carousel advertisement format” on Facebook, had the highest engagement with our target audience and provided a platform to communicate in-depth information about actions people could take to prevent Zika infection. Most organizations have access to someone who specializes in web analytics who can offer their insight.
5. Conduct A/B testing. A/B testing is a way to compare two versions of something to figure out which performs better. A/B testing is used to assess different options of campaign messages or creative concepts in real time and measures which one people actually use more among different channels, such as Facebook and Twitter. The Domestic Zika Campaign tested messages and materials on a number of social and digital platforms to determine which channels our audience preferred. For example, we conducted an A/B test of an existing “Cover Your Body and Use Repellent” advertisement against two new advertisements that presented the “Cover” and “Repellent” content as two separate messages. We conducted a qualitative analysis of user comments on the advertisements and found that users had a negative reaction to the “Cover” action step given the high temperatures in areas with risk of Zika.
6. Refresh your materials. Build in regular measurements to track and observe public response and evaluate public complacency towards your campaign to avoid a decrease in behavioral change outcomes. This can occur for two reasons: 1. Message fatigue, when the attention of your target audience is reduced because they have had repeated exposures to the campaign messages; or 2. Risk fatigue, when your audience no longer receives or responds to messages about the health threat.  You can refresh and adjust your campaign messages without creating new messages from scratch. During the second phase of the Domestic Zika Campaign we revised materials that were targeted towards men by adding images of males with their pregnant partners because we were informed by both women and men that they had become complacent about behaviors men could perform to protect both themselves and their partners from Zika virus infection and transmission.

General Rules of Thumb

7. Do not reinvent the wheel. Know what is available in your organization.  Do you have existing contracts in place to purchase things quickly.  If you have to get permissions to do research, do you have processes in place to navigate them efficiently? As a federal agency, anytime we want to ask more than 9 people a question, we have to get approval from the Office of Management and Budget, which typically takes many months. Knowing this, CDC set up a process called the Health Message Testing System, which is generic request set up in advance with OMB that programs can use to quickly get permission to test a specific message with a specific audience. These approvals can be obtained in days instead of months.
8. Track current events. Scan and track prominent news and social media. Be aware of current events and issues that arise over the course of your campaign. The news media will cover what is timely, and you can take advantage of this coverage to enhance your campaign efforts. During the initial phase of the Domestic Zika Campaign, the proposed use of the chemical pesticide Naled in Puerto Rico created a controversy for public health and government officials on the island. Tracking the controversy allowed us to address this issue by adding a media relations effort to the campaign and provided more opportunities to disseminate the most effective prevention messages.
9. Partner with influential bloggers. Influential bloggers can help enhance and further disseminate your campaign messages, increasing the reach to and saturation of your target audience. One of the main goals of the Domestic Zika Campaign was to amplify CDC’s Zika prevention messages. We worked with a well-known television news celebrity who was pregnant and chose not to attend the summer Olympics in Rio de Janeiro, Brazil. We pitched the story to mom and parenting bloggers and packaged it with CDC-branded messaging and the opportunity for an interview with a CDC subject matter expert. The cost of working with a blogger can vary, and you need to vet them carefully, but there can be great benefits by partnering with someone who is a natural fit for your cause and/or message content AND who is already reaching your target audience.
10. Welcome donated media. Opportunities may arise for you to take advantage of “value-added” or donated media, which can help extend your communication efforts and increase the number of measurement points to evaluate. If you purchase a large number of services, the outside vendor might be willing to add additional advertising or other activities so the campaign will not incur any additional costs. For instance, we acquired donated time for the public service announcements developed by the Domestic Zika Campaign to run in movie theaters in Puerto Rico.

Do you have any other suggestions to improve the quality and timeliness of health marketing and communication campaigns on a budget? Please leave a comment below.

Fred Fridinger is a Senior Health Communications Specialist in the Office of the Associate Director for Communication (OADC). During his 22- year career at CDC, he has worked on various campaigns and communication efforts, including those addressing moderate physical activity and healthy eating, genetics, chronic fatigue syndrome, and Zika prevention. In his current position, he oversees the market research function for OADC, which involves the Porter Novelli Styles and Nielsen Scarborough syndicated data bases.

The Communication Research and Evaluation blog series highlights innovative research and evaluation methods used at CDC to improve behavior change campaigns.