Evaluating Communication Campaigns

Posted on by Fred Fridinger, DrPH, Senior Health Communications Specialist, Office of the Associate Director for Communication

Black man holding belly of his pregnant wife making heart. Pregnant woman and loving husband hugging tummy at home. Heart of hands by multiethnic couple on pregnant belly.

Health communication and marketing campaigns that promote positive behavior change are a cornerstone of public health and behavioral science. Designing and implementing quality campaigns on a tight budget and in an urgent timeframe is a challenge that most health communication professionals share. Research and evaluation are critical for a successful campaign. CDC is using leading research and evaluation methods to develop quality campaigns, while keeping costs low and sticking to tight timelines.

A great example is CDC’s design and implementation of The Domestic Readiness Initiative on Zika Virus Disease, also Domestic Zika Campaign: By the Numbers. Since its launch in early summer 2016, the campaign has generated over 350 million impressions and nearly 555,000 clicks across search, social, and display in the United States. In Puerto Rico, the campaign got over 10 million impressions, and 26,000 clicks on a range of media buys that covered the entire island.known as the Domestic Zika Campaign. This bilingual digital campaign sought to increase Zika awareness among the general population and expectant parents in the United States and its territories. Campaign messages were also developed to change knowledge and attitudes regarding Zika transmission, beliefs about the risks of Zika, and the perceived ability (self-efficacy) to protect oneself against Zika.

CDC identified several methods to get tailored messaging to our target audiences. Based on the work for this campaign, there are 10 methods that can help you reduce costs and improve the efficacy of your communication or health marketing campaign.

Formative Research

  1. Use “mixed” methods or alternative approaches. When feasible, mixed assessment techniques can enhance and complement different types of data collection. Your budget may not allow you to conduct in-depth formative research, but you can use one primary research or evaluation method and then supplement with other methods that are less costly. For example, conducting qualitative focus groups might not account for all of your audience segments, but you could perform a literature search to complement your qualitative data. Additionally, when circumstances dictate, sometimes substituting one evaluation activity for another may be just as effective (or close to it). During the Domestic Zika Campaign we used “triad” interviews, which only involved three participants, when we did not have the time or the budget to conduct a full suite of focus groups, and this was supplemented with previous survey data on closely related topics.
  2. Invest in market data. Public, non-proprietary market data, such as data from the US Census, PEW, and Gallup, allow you to get to know your audience at no cost. Outside marketing firms may also be willing to share “older” proprietary data that may be from a year or two ago at little or no cost. During the Domestic Zika Campaign, secondary data donated from Annenberg surveys(?) were key to our analysis of each of our target demographics’ media habits and informed the mass media and digital media strategies and plans. The cost is your time in building relationships with like-minded partners and analyzing the data for a specific purpose.
  3. Identify alternative data. When traditional surveying of knowledge, attitudes, and behaviors is not feasible, sometimes you can use alternative data (usually less expensive). During the Domestic Zika Campaign, we did not have sufficient time to get approval for our survey from the Office of Management and Budget (OMB) and field a traditional campaign evaluation survey. Instead, to meet our tight timeline, we acquired data about our audiences’ insecticide purchasing behavior in the stores where our advertisements had been placed, and compared them to similar product purchasing data from stores that did not display our advertisements. Building partnerships with organizations who have data, even from related issues, that can offer insights to you is a great long-term investment in both implementing and evaluating programs—especially when we have a shared commitment to protecting people from diseases.

Monitoring & EvaluationDigital Media in an Emergency. During a public health emergency or disease outbreak, even a small number of strategically placed, paid digital media spots can deliver messages to those who need them most. This type of message can often be placed within 24 – 48 hours. During the Zika response, CDC developed and executed behavioral intent studies using data collected by Google Health and Nielsen to study the effect of media exposure on audiences’ intent to take action to protect themselves from Zika. • We monitored exposure to digital campaign banners on the Google Display Network and compared those exposures to an increase in internet search for specific Zika-related terms. • We employed channel-targeted messaging, which helped to amplify CDC’s weekly Zika theme content on Facebook to reach audiences beyond the people who were already following the CDC page. This cost-efficient strategy extended the reach of posts to tens of thousands more people within the specific geographic areas.

  1. Take advantage of social media analytics. Social media platforms like Facebook, Twitter, Instagram, and Google AdWords allow you to target specific geographic areas and have built-in analytics tools that the user has access to at no additional cost. The Domestic Zika Campaign used data analytics to look at the performance of specific advertisements and the corresponding click-through rate to Zika-related campaign websites. Using social media platforms as a primary channel for the campaign allowed for more precise audience targeting, rapid message modifications, and real-time metrics. Using the data, we developed tailored advertisements that engaged key audiences and encouraged them to take preventive actions. For example, one of our treatments, the “carousel advertisement format” on Facebook, had the highest engagement with our target audience and provided a platform to communicate in-depth information about actions people could take to prevent Zika infection. Most organizations have access to someone who specializes in web analytics who can offer their insight.
  2. Conduct A/B testing. A/B testing is a way to compare two versions of something to figure out which performs better. A/B testing is used to assess different options of campaign messages or creative concepts in real time and measures which one people actually use more among different channels, such as Facebook and Twitter. The Domestic Zika Campaign tested messages and materials on a number of social and digital platforms to determine which channels our audience preferred. For example, we conducted an A/B test of an existing “Cover Your Body and Use Repellent” advertisement against two new advertisements that presented the “Cover” and “Repellent” content as two separate messages. We conducted a qualitative analysis of user comments on the advertisements and found that users had a negative reaction to the “Cover” action step given the high temperatures in areas with risk of Zika.
  3. Refresh your materials. Build in regular measurements to track and observe public response and evaluate public complacency towards your campaign to avoid a decrease in behavioral change outcomes. This can occur for two reasons: 1. Message fatigue, when the attention of your target audience is reduced because they have had repeated exposures to the campaign messages; or 2. Risk fatigue, when your audience no longer receives or responds to messages about the health threat.  You can refresh and adjust your campaign messages without creating new messages from scratch. During the second phase of the Domestic Zika Campaign we revised materials that were targeted towards men by adding images of males with their pregnant partners because we were informed by both women and men that they had become complacent about behaviors men could perform to protect both themselves and their partners from Zika virus infection and transmission.

General Rules of Thumb

  1. Do not reinvent the wheel. Know what is available in your organization.  Do you have existing contracts in place to purchase things quickly.  If you have to get permissions to do research, do you have processes in place to navigate them efficiently? As a federal agency, anytime we want to ask more than 9 people a question, we have to get approval from the Office of Management and Budget, which typically takes many months. Knowing this, CDC set up a process called the Health Message Testing System, which is generic request set up in advance with OMB that programs can use to quickly get permission to test a specific message with a specific audience. These approvals can be obtained in days instead of months.
  2. Track current events. Scan and track prominent news and social media. Be aware of current events and issues that arise over the course of your campaign. The news media will cover what is timely, and you can take advantage of this coverage to enhance your campaign efforts. During the initial phase of the Domestic Zika Campaign, the proposed use of the chemical pesticide Naled in Puerto Rico created a controversy for public health and government officials on the island. Tracking the controversy allowed us to address this issue by adding a media relations effort to the campaign and provided more opportunities to disseminate the most effective prevention messages.
  3. Partner with influential bloggers. Influential bloggers can help enhance and further disseminate your campaign messages, increasing the reach to and saturation of your target audience. One of the main goals of the Domestic Zika Campaign was to amplify CDC’s Zika prevention messages. We worked with a well-known television news celebrity who was pregnant and chose not to attend the summer Olympics in Rio de Janeiro, Brazil. We pitched the story to mom and parenting bloggers and packaged it with CDC-branded messaging and the opportunity for an interview with a CDC subject matter expert. The cost of working with a blogger can vary, and you need to vet them carefully, but there can be great benefits by partnering with someone who is a natural fit for your cause and/or message content AND who is already reaching your target audience.
  4. Welcome donated media. Opportunities may arise for you to take advantage of “value-added” or donated media, which can help extend your communication efforts and increase the number of measurement points to evaluate. If you purchase a large number of services, the outside vendor might be willing to add additional advertising or other activities so the campaign will not incur any additional costs. For instance, we acquired donated time for the public service announcements developed by the Domestic Zika Campaign to run in movie theaters in Puerto Rico.

Do you have any other suggestions to improve the quality and timeliness of health marketing and communication campaigns on a budget? Please leave a comment below.

Fred Fridinger is a Senior Health Communications Specialist in the Office of the Associate Director for Communication (OADC). During his 22- year career at CDC, he has worked on various campaigns and communication efforts, including those addressing moderate physical activity and healthy eating, genetics, chronic fatigue syndrome, and Zika prevention. In his current position, he oversees the market research function for OADC, which involves the Porter Novelli Styles and Nielsen Scarborough syndicated data bases.

The Communication Research and Evaluation blog series highlights innovative research and evaluation methods used at CDC to improve behavior change campaigns.

Posted on by Fred Fridinger, DrPH, Senior Health Communications Specialist, Office of the Associate Director for CommunicationTags , , , , , , , , , , ,

11 comments on “Evaluating Communication Campaigns”

Comments listed below are posted by individuals not associated with CDC, unless otherwise stated. These comments do not represent the official views of CDC, and CDC does not guarantee that any information posted by individuals on this site is correct, and disclaims any liability for any loss or damage resulting from reliance on any such information. Read more about our comment policy ».

    Could Dr. Fridinger please communicate to major healthcare providers in the United States that the CDC no longer recommends Graded Exercise Therapy (GET) and Cognitive Behavioral Therapy (CBT) for Chronic Fatigue Syndrome (ME/CFS)?

    GET and CBT are still the norm for ME/CFS clinical care, even though the CDC no longer recommends them.

    Why is the CDC unwilling or unable to properly communicate their own treatment changes for ME/CFS?

    This essay claims that CDC was able to develop an effective communications campaign for Zika. I’m willing to accept that claim.

    So why is CDC unable to do the same for Myalgic Encephalomyelitis (ME)? Thirty years after CDC renamed ME to the demeaning label “chronic fatigue syndrome”, most physicians believe ME to be psychological, that “fear of exercise” keeps us sick, as even CDC’s own Dr Elizabeth Unger admits. CDC has made no effort to change this perception or stop the abuse of ME patients by physicians, social service agencies, and society at large.

    Recently CDC quietly dropped graded exercise therapy from its “ME/CFS” treatment recommendations but no one outside the ME patient community knows this. Meanwhile the CFS Advisory Committee is struggling to “reinvent the wheel” and trying to figure out how to inform physicians about proper diagnosis and treatment.

    In the face of CDC competence in regards to Zika, and incompetence in regards to ME, it is difficult to avoid the conclusion that there is an unwritten policy of keeping physicians and the public deliberately misinformed. Please prove me wrong.

    In my opinion, publishing in specific groups and with large numbers of people would help as well.

    It’s amazing to see how efficient and fast, important information gets out to the general public. The CDC website is always a great site to get most up-to- date health information but to learn about so many ways that health communication through marketing campaigns happens is very informational. As mentioned above with the Domestic Zika Campaign that was launched in 2016, the numbers of 350 million impressions and 555,000 clicks through various avenues were phenomenal. Through CDC’s various methods to increase efficacy to communicate to the general public on any health issues not just imperative but also can prevent further spread of infections, diseases and helps the healthcare industry treat any population faster and correctly. Great Information!!

    Health communication is the study and practice of communicating promotional health information, such as in public health campaigns, health education, and between doctor and patient. The purpose of disseminating health information is to influence personal health choices by improving health literacy.

    Hi Dr. Fridinger. Has there been an update to the Cultural Competency piece you did on Latinos.

    Has there been any updates to the Latino Cultural Competence article you wrote a while ago?

    Do you have a recommendation as to what % of the budget should be devoted to evaluation for a youth campaign re: prevention?

    You’ve made some really good points there. I checked on the net to find out more about the issue and found most people will go along with your views on this site.

    This Articla is very good it givess so much information for the students who want to learn A/B testing. very nice article.

Post a Comment

Your email address will not be published. Required fields are marked *

All comments posted become a part of the public domain, and users are responsible for their comments. This is a moderated site and your comments will be reviewed before they are posted. Read more about our comment policy »

Page last reviewed: April 2, 2018
Page last updated: April 2, 2018