Ask the Expert: LCDR Erin Nichols, Verbal Autopsy

[Editor’s Note: NCHS’s International Statistics Program collaborates with domestic and international partners in many public health projects around the world. We spoke with epidemiologist LCDR Erin Nichols about one of the program’s more ambitious projects: verbal autopsy.]

 

Q. How did the verbal autopsy project come about, and who you will be working with?

A. Very broadly, we started this project within the scope of the International Statistics Program’s civil registration and vital statistics improvement project. Part of the project is focusing on improving the quality of cause-of-death information for countries to use for program planning and evaluation. We can do the standard training with physicians and use the WHO [World Health Organization] certificate of medical cause of death for deaths that are overseen by physicians. But what can we do with all the other deaths that aren’t seen by a doctor, or don’t take place in a hospital? For those other deaths, we go to verbal autopsy.

Verbal autopsy involves an interview with a caregiver of a deceased person to get information about signs and symptoms experienced by the deceased around the time of death. This information is used to get a probable cause of death. I think the first verbal autopsies were done 30 to 40 years ago using nonstandard forms. The method has evolved over the years, and really in the last 10–15 years, there’s been a push to standardize the practice so that you can compare data, and so people have some indication of what tools have been tested and which ones are recommended. In 2007, WHO came out with verbal autopsy standards. The interviews took about 45 minutes and required physicians to review the results and determine cause of death.

Until now, verbal autopsy has really been used in areas of research, in demographic surveillance sites, or where there’s a specific targeted interest, like maternal deaths. When you’re thinking about civil registration, though, you’re dealing with a large number of deaths and a continuous process, so a research-type instrument is not appropriate, not for routine use.

So WHO and others have made an effort to simplify the 2007 instrument and make it compatible with mobile devices and an automated analysis program. They invited experts in verbal autopsy from around the world to come to Geneva [Switzerland] and talk about their experiences. Working with the experts, they began to cut [out] questions that complicated the process and really didn’t contribute to identifying causes of death, and questions that, even if they were useful in identifying cause of death, really weren’t something that the respondent could answer about somebody else.

 

Q. Questions that created more problems than they solved?

A. Exactly. So in July 2012, WHO released a new verbal autopsy candidate for routine use. And now we’re in a testing period to try out the new instrument. We’ll spend the next one to two years testing it, getting feedback, and then ultimately making updates and, hopefully, releasing a standard recommended instrument for routine use.

So, backtrack to our project. We work very closely, through work on the International Classification of Diseases, with the same partners at WHO that are doing this verbal autopsy work. Through our civil registration and vital statistics improvement project, we’re able to provide field-testing sites for the 2012 verbal autopsy instrument. We’re able to work very closely with WHO in planning the field testing to make sure we’re asking, and aiming to answer, the right questions out there.

Most of the testing will be quantitative. They’re going to combine the 2007 instrument with the 2012 release candidate. You will have all indicators from both instruments and administer it as one interview. Then, when you analyze the results, you can take the results from the 2007 indicators with physician review, and compare those with the results from the automated analysis using the 2012 indicators.

I went to Geneva back in February to meet with the people that are working on the combined instrument, to provide the perspective of what’s going to happen in the field. They’ve got a draft of the combined instrument, which we are going to use in Malawi where we’re assisting with a PEPFAR [President’s Emergency Plan for AIDS Relief]-funded project through the CDC Malawi office, together with USAID [U.S. Agency for International Development]. It’s the first civil registration and vital statistics project funded by PEPFAR, so we’re hoping this will be a good model for future projects.

 

Q. What is PEPFAR?

A. PEPFAR is the U.S. President’s Emergency Plan for AIDS Relief. It was created in 2003, and was focused on HIV/AIDS treatment and prevention. Now we’re 10 years on, and they’re really broadening the program toward improving health systems. We’ve got good treatment regimens; we’ve got good coverage of people getting treatment. Now, what can we leave behind that’s more of a systems-based approach?

Of course, having information on mortality and on what people are dying from helps the HIV/AIDS programs monitor their progress, so our partners in Malawi have received funding to support this project to strengthen Malawi’s vital statistics system, with a focus on information coming from civil registration.

In the first part of the Malawi project, they’ll be using SAVVY, which is Sample Vital Registration With Verbal Autopsy, as a sample-based means to collect mortality information. It’s very resource-intensive, but you get representative data on death using verbal autopsy. In Malawi, we’ll have representative data for each of the five health zones. Then, in the second part of the project, we’ll work to expand the SAVVY into full civil registration in Malawi.

We’re partnering with the Census Bureau and MEASURE [Monitoring and Evaluation to Assess and Use Results] Evaluation, who are being coordinated through the USAID portion of this project in Malawi, and they’re going to conduct the SAVVY survey starting this year. MEASURE Evaluation has done a lot of work in the past to develop SAVVY methods, which traditionally use the longer 2007 WHO VA [verbal autopsy] instrument with physician review. They agreed to use the combined instrument in their survey. So then we’ll be able to quantitatively compare the results of the 2007 and 2012 verbal autopsy instruments to see if the simplified instrument is acceptable for use.

 

Q. Are you testing the verbal autopsy instrument anywhere else?

A. Yes. We are also working in Kenya, where we have a project with their Civil Registration Department and their Ministry of Health. Kenya is very interested in verbal autopsy, so we want to look to recommend this instrument there. We will test methods for improving birth and death registration, starting in one district in Kenya.

The Kenya Medical Research Institute [KEMRI], together with the CDC, has a demographic surveillance site in western Kenya where they’ve been doing verbal autopsy for over 10 years. They are cutting-edge on verbal autopsy methods—always trying something to make the process more efficient, better. So they’re interested in testing this new instrument as well.

 

Q. They’ve been using their own homegrown version for the last 10 years?

A. The version they have now is based on the WHO 2007 questionnaire. But because they’re a specific research surveillance site, they have a lot of interest in tracking certain programmatic areas where they have projects, so it’s a little bit tweaked. They’re in the process of combining their questionnaire with the 2012 instrument as well. They’re going to use the same methods where they add in the 2012 indicators, run the whole questionnaire, and then compare the results.

In the Kenya surveillance site, they recently moved to using an automated analysis program called InterVA. They’ve already shifted to a version of InterVA that was developed for the 2007 instrument. And now they’re going to compare the automated 2007 results with the automated 2012 results. This will provide another quantitative comparison of the instrument. With this testing, we’re going to work closely with them to see how it works in the surveillance site first, and then look to implement the instrument in our demonstration district for the civil registration project.

On top of the quantitative testing, we will also be doing cognitive testing in Kenya. This is a unique and exciting opportunity for collaboration with NCHS’s Questionnaire Design Research Laboratory [QDRL]—and to our knowledge, this type of testing has never been done formally for verbal autopsy. We want to look at whether the questions we are intending to ask are being understood by respondents and how the instrument’s structure might affect responses. Of course, much of verbal autopsy is context-specific, but we think that the testing will provide good insight to add to the development of the instrument, as well as insight specifically for application in Kenya.

 

Q. Will the cognitive interviewing encompass both the 2007 and 2012 versions?

A. Both—the testing will be based on the combined instrument. What we’re in the process of doing now is selecting the specific elements that we’ll target for the cognitive interviewing, because we won’t be able to test all questions. When I went to Geneva in February, a lot of questions emerged about certain indicators, so we’ll include those in the testing.

What’s really neat about this project is that so many people are interested in the cognitive testing, like the [CDC] injury group in Atlanta. The 2012 instrument includes a new injury section, so we can test that cognitively. We’ll also reach out to others for input—to CDC partners and others that have used verbal autopsy before or will likely look to use the 2012 verbal autopsy instrument. We want to see what their questions are. What problems do they have?

So once we identify all key elements to test, we’ll be able to determine how best to test those elements. We are excited about having the 2007 questions, to see if added information from that version supports the respondents’ answers.

For example, a lot of what was cut out of the 2012 questionnaire has to do with duration of symptoms. For a physician’s review, duration is very important in determining if the symptom was related to a given cause of death. But it can be challenging to incorporate into the computer programming for the automated analysis. You have to dichotomize all the information going into this automated process, so you have to really think about what are the specific cutoffs going to be for a given symptom.

So where the duration was cut or dichotomized in the 2012 instrument, we can further explore cognitively if and how that might affect responses. Then we will have a better idea of what questions should be asked and how.

Something else we have to keep in mind for the cognitive testing is that the published version of the verbal autopsy instrument is in English—that’s how it’s released to everybody. But verbal autopsy is almost never implemented in English—it’s translated. Studying questionnaire design, we can get hung up on terminology. But the key here is not so much what a question says in English, but that the proper intention of the question is fully understood by those who are translating it and customizing it for the local culture. So rather than making sure we have the absolute correct wording in the English, there is really an emphasis on the training component and instructions for the translators and the interviewers, to help them accurately capture the intention of the question. We think that, from the cognitive testing, we’ll be able to provide a lot of insight into what that guidance should be for the interviewers and translators.

 

Q. That’s a lot of work.

A. [Laughs] That’s a lot of work! So we’re grateful that our SAVVY partners in Malawi and our KEMRI/CDC partners in Kenya are interested and willing to test the 2012 instrument. The stars are aligned with everything right now to conduct this testing.

A lot of our efforts have been just keeping in touch with everyone, connecting, leveraging ongoing work where we can. I’m serving as the go-between with the work going on at the WHO to combine the instruments, and the SAVVY team in Malawi, to make sure we get it to them as they’re getting ready to develop their data collection platform. The SAVVY team’s input will be really useful because they know what the 2007 indicators are, and when they see the combined questionnaire, they can raise any flags like, “Hey, this [question] isn’t in there now, we’re a little concerned about this.” It will be a good way to cross-check the process that was done in an office, in preparation for application in the field.

And then in Kenya, we’re working with the QDRL to do the cognitive interviewing, and the KEMRI/CDC team that will do their combination-instrument combination and testing. Then our task will be to draw all of this information together and, hopefully, be able to make good recommendations for the countries that we’re working with. Several countries are extraordinarily interested in verbal autopsy, but, of course, they always ask, “Is it WHO-endorsed?” We want to be able to recommend to countries a tested, validated standard instrument for verbal autopsy.

 

Q. Is it mainly African countries, or do you see this having global implications?

A. Definitely global. Most of our project work is in African countries right now, but countries around the world are interested in verbal autopsy. I know of other work going on in Brazil, India, and China, just to name a few. We are also in communication with colleagues in India. There’s a U.S.–India Health Initiative organized out of the Department of State, and one component of the initiative is focused on their vital registration system. They have an interest in improving the verbal autopsy work that they’ve been doing. Right now, India uses basically an open narrative, where you say, “Tell me what happened,” the respondent tells you, and then, from that information, a physician makes a decision about the cause of death. There’s no standardization. The narrative, when physicians are reviewing it, is by far the most important component. But they’re interested in exploring other global standards, as well as looking at the automated analysis, to free up the physicians.

 

Q. When do you actually start in the field?

A. In Malawi, they are aiming to begin pilot testing in May, so fingers crossed, data collection will start this summer sometime. They’ll be running their field work for two years, so we’ll check in every three to six months to look at the data. In Kenya, we’re also hoping to start testing the instrument this summer, and we’re hoping to do the cognitive interviewing in September. So hopefully by December, we’ll have some good results from what’s going on in Kenya.

A lot of work is being done to improve verbal autopsy methods right now—looking at different questionnaires and different analysis methods. We’ve put a lot of effort into researching this work, pulling all the parties together and figuring out where we are and what needs to be done. We’ve done a lot of good preliminary work to develop our projects, so hopefully we will start seeing some good results soon.