Overreporting of Deaths From Coronary Heart Disease in New York City Hospitals, 2003

ORIGINAL RESEARCH
Overreporting of Deaths From Coronary Heart Disease in New York City Hospitals, 2003

Reena Agarwal, MD, MPH; Jennifer M. Norton, PhD; Kevin Konty, MS, MA; Regina Zimmerman, PhD, MPH; Maleeka Glover, ScD; Akaki Lekiachvili, PhD; Henraya McGruder, PhD; Ann Malarcher, PhD; Michele Casper, PhD; George A.  Mensah, MD; Lorna Thorpe, PhD

Suggested citation for this article: Agarwal R, Norton JM, Konty K, Zimmerman R, Glover M, Lekiachvili A, et al. Overreporting of deaths from coronary heart disease in New York City hospitals, 2003. Prev Chronic Dis 2010;7(3). http://www.cdc.gov/pcd/issues/2010/
may/09_0086.htm. Accessed [date].

PEER REVIEWED

Abstract

Introduction
New York City has one of the highest reported death rates from coronary heart disease in the United States. We sought to measure the accuracy of this rate by examining death certificates.

Methods
We conducted a cross-sectional validation study by using a random sample of death certificates that recorded in-hospital deaths in New York City from January through June 2003, stratified by neighborhoods with low, medium, and high coronary heart disease death rates. We abstracted data from hospital records, and an independent, blinded medical team reviewed these data to validate cause of death. We computed a comparability ratio (coronary heart disease deaths recorded on death certificates divided by validated coronary heart disease deaths) to quantify agreement between death certificate determination and clinical judgment.

Results
Of 491 sampled death certificates for in-hospital deaths, medical charts were abstracted and reviewed by the expert panel for 444 (90%). The comparability ratio for coronary heart disease deaths among decedents aged 35 to 74 years was 1.51, indicating that death certificates overestimated coronary heart disease deaths in this age group by 51%. The comparability ratio increased with age to 1.94 for decedents aged 75 to 84 years and to 2.37 for decedents aged 85 years or older.

Conclusion
Coronary heart disease appears to be substantially overreported as a cause of death in New York City among in-hospital deaths.

Back to top

Introduction

Coronary heart disease (CHD) is the leading cause of death for adults in the United States, and stroke ranks third (1). In New York City, an unusual pattern of high CHD death rates and low stroke death rates has been observed; the age-adjusted CHD death rate in 2003 was 1.7 times the national rate, and the age-adjusted stroke death rate was half the national rate (1,2). This pattern is unexpected, given that risk factors for CHD and stroke are similar and that the prevalence in New York City of most common CHD risk factors, such as hypertension, hyperlipidemia, obesity, and smoking, is similar to or lower than that in the rest of the country (3). New York City CHD death rates have been consistently higher than national CHD death rates for more than 3 decades despite steady CHD death rate declines nationally and in New York City (Figure).

Figure. Age-adjusted deaths from coronary heart disease in New York City (NYC) versus the United States overall. “ICD” indicates the revisions of the International Classification of Diseases that were used to categorize cause of death. US data for 1950 through 2002 from National Heart, Lung, and Blood Institute, 2002. US data for 2003 through 2006 from Centers for Disease Control and Prevention. NYC data from NYC Department of Health and Mental Hygiene, Bureau of Vital Statistics. NYC population data from 1971 through 2003 census estimates. NYC population data from 1961 through 1969 from linear interpolation. [A tabular version of this figure is also available.]

Misreporting cause of death on death certificates may be contributing to New York City’s observed high CHD death rate. Studies have suggested that such misreporting may be common. A study of 2 Texas military hospitals found that 37% of death certificates reported a different cause of death than did autopsy (4). A Swedish study found that 54% of death certificates reported a different cause of death than did chart review (5). The level of misreporting varies by cause of death. For CHD, a British study found that death certificates have a low sensitivity for CHD deaths compared with autopsy findings (6). A study of Framingham Heart Study participants found that death certificates attributed 24% more deaths to CHD than did a physician panel that reviewed medical records (7). The Atherosclerosis Risk in Communities (ARIC) study also found that death certificates overestimated CHD deaths by 20% compared with a physician review panel (8).

We sought to determine whether local physician reporting patterns of CHD on death certificates contribute to the observed pattern of high-CHD, low-stroke death rates observed in New York City. Our investigation was designed to determine the accuracy of death certificate reporting of CHD as a cause of death in New York City by using methods employed by the ARIC study in other jurisdictions. We did not directly address accuracy of death certificate reporting of stroke in this analysis.

Back to top

Methods

Study design and population

We developed the study design by using the methods of the ARIC study, which evaluated CHD reporting on death certificates in 4 geographically distinct US communities by using a blinded medical record review process (8). A completed “determination of research status” form was submitted to the Centers for Disease Control and Prevention (CDC), which determined that this study would not require institutional review board approval. We assessed New York City death certificates dated January through June 2003, including those for which the decedent 1) was a resident of New York City, 2) died in one of the city’s 70 hospitals, and 3) had an underlying cause of death on his or her death certificate that corresponded to a subset of codes associated with CHD or stroke from the International Classification of Diseases, 10th revision (ICD-10) and corresponding to the ICD-9 codes used in the ARIC validation study (Table 1). A data set with 13,144 eligible death certificates, including 7,674 for in-hospital deaths, was obtained from the New York City Department of Health and Mental Hygiene (DOHMH), Bureau of Vital Statistics. We took a stratified random sample based on New York City neighborhoods with low, medium, or high age-adjusted CHD death rates in 2001; each stratum contained roughly one-third of the sampled population. This study was restricted to 491 in-hospital deaths to maximize feasibility of data collection. In-hospital deaths were categorized as inpatient, emergency department (ED), outpatient, and dead on arrival (DOA).

Data collection

Hospitals provided medical records for the sampled decedents. We abstracted information from each record and entered it into a standardized computer-based abstraction form developed by using EpiInfo version 3.4.1 (CDC, Atlanta, Georgia). This form contained pre-populated fields for demographic data reported on the death certificates. Information abstracted from the medical chart included time from onset of symptoms to death, presence of chest pain, history of CHD, and medication use. We photocopied electrocardiogram results (ECGs), test results for cardiac enzymes, and medical discharge summaries.

External review

Two external teams were assembled, each consisting of 2 physician epidemiologists, trained and certified in the ARIC protocol for validation of CHD events. They were blinded to the cause of death listed on the death certificate and used case summary reports generated from the medical record abstraction, ECGs, and medical discharge summaries to answer 5 yes/no questions about each case: 1) Was there a known nonatherosclerotic or noncardiac process or event that was probably lethal?, 2) Was there a definite myocardial infarction (MI) for which the patient was hospitalized within 4 weeks of death?, 3) Was there a probable MI for which the patient was hospitalized within 4 weeks of death?, 4) Was there a history of chest pain within 72 hours of death?, and 5) Was there a history of ever having had chronic ischemic heart disease such as coronary insufficiency or angina? When initial responses to questions differed within each team, the 2 reviewers discussed the case until they agreed on a final determination.

Case definitions

Death certificate CHD deaths were defined as ICD-10 codes I20-I25 and I51.6. Death certificate non-CHD deaths were defined as all other ICD-10 codes included in the study (Table 1). We used an algorithm based on the ARIC methods, incorporating the reviewer responses to the 5 yes/no questions described above, to develop 2 definitions of CHD: definite CHD and possible CHD. Definite CHD was defined as 1) no known nonatherosclerotic or noncardiac process or event that was probably lethal and 2) at least 1 of the following conditions: definite MI for which the patient was hospitalized within 4 weeks of death, history of chest pain within 72 hours of death, or history of chronic ischemic heart disease such as coronary insufficiency or angina pectoris. The definite CHD validation definition relies heavily on information available in the medical record. To account for the possibility that a death may be due to CHD even if relevant information is not clearly documented in the chart, we used a second case definition, possible CHD. Possible CHD was defined as 1) no known nonatherosclerotic or noncardiac process or event that was probably lethal and 2) a death certificate with a consistent underlying cause of death (ICD-10 codes I20-I25 and I51.6). This definition is more sensitive, excluding only those deaths with a clearly recorded cause other than CHD.

Data analysis

The goal of data analysis was to estimate the degree to which CHD deaths were reported on death certificates when measured against the standard of external medical chart review. To obtain this measurement, we calculated comparability ratios (CRs) as the number of CHD deaths defined by the death certificate divided by the number of CHD deaths defined by review of the medical record. We also calculated sensitivity, false-positive rate (FPR), and positive predictive value (PPV).

The final data set contained records that were abstracted and validated (n = 444). Observations were assigned initial weights equal to their inverse probability of selection, which varied by strata. The target population consisted of all New York City in-hospital decedents with sampled ICD-10 codes (n = 7,800) who died during the study period. We used post-stratification weighting to account for differences in race/ethnicity and sex distributions between the sample and the target population. To account for the complex sampling design, we computed variance estimates for CRs by using a jackknife estimator for stratified samples (9).

For direct comparison to the ARIC study of inpatient deaths (excluding ED, outpatient, and DOA deaths), we also calculated survey weights and outcome measures for the inpatient subpopulation. Findings were similar for the inpatient-only and comprehensive samples; thus, detailed results are presented only for the larger in-hospital sample.

Back to top

Results

Of the 491 eligible cases, 444 records (90%) were abstracted from hospital charts and assessed by the reviewers. Charts were unavailable for the remaining 47 cases. Of the 444 decedents, 345 were inpatients; 70 were ED patients or outpatients; 23 were DOA, and 6 died in other or unknown places in the hospital (Table 2). Most were women (54%), aged 75 years or older (66%), non-Hispanic white (55%), and died in a private hospital (89%).

Reviewer disagreement before adjudication on each of the 5 validation questions ranged from 5% to 31%. The sensitivity of death certificates for definite CHD deaths was 0.87, the FPR was 0.66, and the PPV was 0.46 (Table 3). The overall CR for definite CHD was 1.91 (95% confidence interval [CI], 1.59-2.23). The CR increased with age, from 1.51 for decedents aged 35 to 74 years to 2.37 for decedents aged 85 years or older. The CR was 1.83 for women and 2.01 for men; it was 2.08 for whites, 2.14 for blacks, and 1.30 for Hispanics. When the broader possible CHD definition was used, the sensitivity was 0.91, the PPV was 0.66, and the FPR was 0.54. The CR for possible CHD deaths was 1.37 (95% CI, 1.21-1.53).

When data were stratified by neighborhoods with low, medium, and high CHD death rates, the CR increased progressively from low to high strata. The CR for definite CHD in the low stratum was 1.82 (95% CI, 1.40-2.24), in the medium stratum was 2.06 (95% CI, 1.67-2.45), and in the high stratum was 2.79 (95% CI, 2.43-3.15). This trend remained consistent in the inpatient population (CR range, 1.71-2.80) and when using the “possible CHD” validation definition for both the total in-hospital population (CR, 1.28-1.99) and the inpatient-only population (CR, 1.30-2.36).

Back to top

Discussion

These results demonstrate substantial overreporting of CHD as a cause of death on death certificates in New York City for in-hospital deaths, when measured against the standard of medical record review. The CR for definite CHD was 1.51 for in-hospital decedents and 1.33 for inpatient decedents (in-hospital deaths excluding DOA, outpatient, and ED deaths) aged 35 to 74 years. The inpatient population is comparable to the inpatient population in the ARIC study, which examined deaths among decedents aged 35 to 74 years in 4 other sites across the country. The ARIC study found modest underreporting of CHD deaths on death certificates compared with chart review by using the same validation definition for definite CHD (CR, 0.9). These results indicate that overreporting of CHD on death certificates may contribute to the elevated CHD death rates observed in New York City.

The study design was based on the methods of the ARIC validation study but differs from the ARIC study in 3 ways. First, the ARIC study was conducted from 1987 through 1995, when ICD-9 was used to code cause of death, whereas this study was conducted in 2003 by using ICD-10 codes. Given the high correspondence between ICD-9 and ICD-10 for CHD (10), the change in classification system is unlikely to account for the differences in the CRs between ARIC and this study. Second, stroke deaths reported on death certificates were not included in ARIC but were included in our study. Including stroke deaths could have resulted in a greater opportunity for false or true negatives. False negatives would have resulted in a lower CR. However, our sample included only a small number of stroke deaths, and all were classified as true negatives. Therefore, the inclusion of stroke deaths did not affect our comparison with ARIC. Third, this study included a high proportion of decedents aged 75 years or older, many of whom had multiple chronic medical problems; the presence of these comorbidities made validation especially difficult, as evidenced by the range of initial disagreement between the reviewers on the 5 validation questions. Difficulty in validating cause of death was also due to conflicting, sparse, or missing information in hospital charts. When reviewers were uncertain how to answer a question, they chose “no,” frequently resulting in a validated “No CHD” classification. This in turn yielded fewer validated CHD deaths, possibly inflating the CR.

To our knowledge, this is the first study to examine CHD reporting on in-hospital death certificates in New York City. The findings have implications for public health and vital registration practice. Further work is needed to better understand patterns of death certificate completion for in-hospital deaths. Providers who are most knowledgeable about the patient do not always complete the death certificate; that task may be assigned to residents or fellows who are not properly instructed in completion, leading to inaccurate reporting (11). In addition, providers who complete the death certificate may not have all relevant patient information available at that time. In particular, physicians in the ED have limited information at the time of death certificate completion. External reviewers may have had the benefit of pathology or laboratory reports unavailable to the provider at the time of death certificate completion. Lack of information about DOA, outpatient, and ED decedents may account for the difference between inpatient and in-hospital CRs.

Other reasons for CHD overreporting on death certificates may be more specific to New York City. New York City has a central registration process, but the rest of the state has approximately 1,500 local registrars, similar to most other health jurisdictions. Therefore, any bias in reporting, such as misunderstanding regarding how death certificates should be completed, may be compounded. For example, in the past the New York City burial desk rejected death certificates for improper completion of cause of death. This rejection process no longer occurs, but the fear of having a death certificate rejected may still lead many physicians to complete certificates with common and “acceptable” causes of death, such as CHD. Second, New York City requires death certificates to be registered within 72 hours of death. This time pressure may result in more frequent reporting of certain causes of death that are easier to assume in a decedent with many comorbidities, such as older decedents. Third, the large number of teaching hospitals in New York City may compound the problem of providers who are less familiar with the decedent’s medical history being responsible for completing death certificates.

This study highlights the possibility that New York City’s observed high rate of CHD may be partly due to misreporting of CHD on death certificates. Death certificate accuracy may be improved by physician education. The city’s Bureau of Vital Statistics has developed an online tool to educate providers in all phases of their careers about death certificate completion, and other education materials are being developed (12). As the success of these efforts are evaluated, it will be important to monitor trends in the reporting of CHD deaths on death certificates and to assess any changes in the CHD death rates that could be due to changes in reporting practices. Meanwhile, CHD is the leading cause of death in New York City, and efforts to reduce its burden remain a top public health priority. Having accurate data will help DOHMH meet its mission to protect and promote the health of all New Yorkers.

Back to top

Acknowledgments

Data collection for this study was partially supported by the Centers for Disease Control and Prevention, Epidemiology Program Office.

Back to top

Author Information

Corresponding Author: Reena Agarwal, MD, MPH, Division of General Internal Medicine, Montefiore Medical Center, 111 E 210th St, Bronx, NY 10467. Telephone: 718-944-3840. E-mail: ragarwal@montefiore.org. At the time of the study, Dr Agarwal was affiliated with the New York City Department of Health and Mental Hygiene, New York, New York.

Author Affiliations: Jennifer M. Norton, Kevin Konty, Regina Zimmerman, New York City Department of Health and Mental Hygiene, New York, New York; Maleeka Glover, Akaki Lekiachvili, Henraya McGruder, Ann Malarcher, Michele Casper, Centers for Disease Control and Prevention, Atlanta, Georgia; George A. Mensah, PepsiCo, Purchase, New York; Lorna Thorpe, CUNY School of Public Health at Hunter College, New York, New York. At the time of the study, Dr Mensah was affiliated with the Centers for Disease Control and Prevention, Atlanta, Georgia, and Dr Thorpe was affiliated with the New York City Department of Health and Mental Hygiene, New York, New York.

George A. Mensah, Director, Heart Health and Global Health Policy Global Research and Development PepsiCo 700 Anderson Hill Road, Bldg 6-2 Purchase, NY 10577

Back to top

References

1. Hoyert DL, Heron MP, Murphy SL, Kung H. Deaths: final data for 2003. Natl Vital Stat Rep 2006;54(13):1-120.
2. New York City Department of Health and Mental Hygiene. Annual mortality data file – 2003. New York (NY): Bureau of Vital Statistics; 2003.
3. Gwynn RC, Garg RK, Kerker BD, Frieden TR, Thorpe LE. Contributions of a local health examination survey to the surveillance of chronic and infectious diseases in New York City. Am J Public Health 2009;99(1):152-9.
4. Selby DM, Clark B, Cina SJ. Accuracy of death certification in two tertiary care military hospitals. Mil Med 1999;164(12):897-9.
5. Johansson LA, Westerling R. Comparing hospital discharge records with death certificates: can the differences be explained? J Epidemiol Community Health 2002;56(4):301-8.
6. Sington JD, Cottrell BJ. Analysis of the sensitivity of death certificates in 440 hospital deaths: a comparison with necropsy findings. J Clin Pathol 2002;55(7):499-502.
7. Lloyd-Jones DM, Martin DO, Larson MG, Levy D. Accuracy of death certificates for coding coronary heart disease as the cause of death. Ann Intern Med 1998;129(12):1020-6.
8. Coady SA, Sorlie PD, Cooper LS, Folsom AR, Rosamond WA, Conwill DE. Validation of death certificate diagnosis for coronary heart disease: the Atherosclerosis Risk in Communities (ARIC) Study. J Clin Epidemiol 2001;54(1):40-50.
9. Berger YG. A jackknife variance estimator for unistage stratified samples with unequal probabilities. Biometrika 2007;94(4):953-64.
10. Final comparability ratios for 113 selected causes of death. National Center for Health Statistics. ftp://ftp.cdc.gov/pub/health_statistics/NCHS/Datasets/Comparability/ icd9_icd10/. Accessed January 24, 2007.
11. Lakkireddy DR, Gowda MS, Murray CW, Basarakodu KR, Vacek JL. Death certificate completion: how well are physicians trained and are cardiovascular causes overstated? Am J Med 2004;117(7):492-8.
12. Improving cause of death reporting. New York City Department of Health and Mental Hygiene. http://www.nyc.gov/html/doh/media/video/icdr/index.html. Accessed December 11, 2009.

Back to top

Tables

Table 1. ICD-9 and ICD-10 Codes Used to Report CHD as a Cause of Death on Death Certificates, New York City, 2003

Abbreviations: ICD-9, International Classification of Diseases, 9th revision; ICD-10, International Classification of Diseases, 10th revision; CHD, coronary heart disease; ARIC, Atherosclerosis Risk in Communities; MI, myocardial infarction; IHD, ischemic heart disease.
^a The ARIC study evaluated coronary heart disease reporting on death certificates in 4 geographically distinct US communities using a blinded medical record review process (8).
^b No analogous code.

Table 2. Characteristics of Decedents for Whom Cause of Death Was Reported as CHD on Death Certificates, New York City, 2003

Abbreviations: CHD, coronary heart disease; ED, emergency department; DOA, dead on arrival; ICD-10, International Classification of Diseases, 10th revision.
^a Percentages may not total 100 because of rounding.
^b More than 1 ICD code may be recorded per record.

Table 3. Validation Measures of Death Certificate Accuracy for Reporting CHD, New York City, 2003

Abbreviation: CHD, coronary heart disease; CI, confidence interval.
^a CHD defined by death certificate and by validation divided by CHD defined by validation.
^b CHD defined by death certificate and by validation divided by CHD defined by death certificate.
^c CHD defined by death certificate but not by validation divided by all death certificates validated as not CHD.
^d CHD defined by death certificate divided by CHD deaths defined by validation.
^e Defined as 1) no known nonatherosclerotic or noncardiac process or event that was probably lethal and 2) at least 1 of the following conditions: definite myocardial infarction for which the patient was hospitalized within 4 weeks of death, history of chest pain within 72 hours of death, or history of chronic ischemic heart disease such as coronary insufficiency or angina pectoris.
^f Inpatient sample is equivalent to the in-hospital sample excluding outpatient, emergency department, and dead on arrival deaths.
^g Defined as 1) no known nonatherosclerotic or noncardiac process or event that was probably lethal and 2) a death certificate with a consistent underlying cause of death.

Back to top