How Can Evidence Synthesis be Conducted at the Speed of a Pandemic?

Since December, 2019, SARS-CoV-2, the virus that causes COVID-19, has rapidly spread around the world. The global pandemic has led to numerous scientific publications in basic, clinical and public health science and increased pressure to act on evidence as it emerges. Typically, research is done to inform decisions over time and is disseminated to the public and policy makers. However, the urgency of COVID-19 has led to an explosion of scientific output.

We conducted a search in PubMed for the term COVID-19 and found more than 7,000 articles published between January 1 and April 27, 2020. This does not capture the thousands of studies shared on preprint servers. The speed at which new information is now available about COVID-19 is far outpacing our ability to synthesize the cumulative evidence in a timely fashion.

A traditional approach to synthesize evidence is the use of systematic reviews. Ideally, systematic reviews have clear criteria for which studies are included and excluded, and – when conducted well – should assess the quality of each study. Systematic reviews can incorporate information from observational studies, non-randomized trials, and randomized controlled trials to clarify existing evidence about a certain topic. Some reviews use statistical methods of meta-analysis to assess and summarize the results of multiple studies.

We conducted a quick PubMed search on April 12 and found at least 28 meta-analyses and systematic reviews about COVID-19.

The topics of these studies were wide-ranging, and included:

• Understanding basic epidemiological evidence for COVID-19
• Assessing the utility of different types of masks
• Assessing the value of closing schools
• Describing efficacy and safety of certain drugs in the treatment of COVID-19

Only a few of these studies indicate that the authors used standardized tools for quality measures to guide the review process, which is an important way of standardizing results presented in systematic reviews and meta-analyses. Only 1 meta-analysis used GRADE quality measures. An additional 5 studies used PRISMA guidelines to guide the review process and report their results.

Currently, systematic reviews of COVID-19 topics are being published at a very rapid pace, running the risk of synthesizing data from flawed, poorly designed studies, or studies with overlapping data. In fact, just before we published this blog, we conducted a more recent search on April 27 and found at least 90 published systematic reviews or meta-analyses. Systematic review protocol registries can help to reduce duplication of work and increase transparency during evidence synthesis. PROSPERO, a prospective registry of systematic reviews, contains information on well over 600 protocols for systematic reviews of human studies relevant to COVID-19.

Rada (2020) described specific concerns with current methods used for systematic reviews, including restricted scope of the review, duplication of efforts and inefficiency in developing these reviews, potential for the reviews to become obsolete quickly, and poor quality (e.g., lack of rigor and transparency). Thus, it is necessary to consider innovative ways to generate high-quality evidence as rapidly as possible.

There are several ways to develop the best available evidence for effective prevention and treatment of COVID-19 and to improve the generation of high-quality synthesized evidence without sacrificing quality. These include:

• Conducting collaborative studies with full data sharing
• Promoting rapid publication with proper peer-review
• Developing ongoing “living evidence synthesis” approaches (as suggested by Rada et al.) that compile evidence in real-time using an open-access platform available to researchers

Quality evidence synthesis is more important than ever. To save the most lives, we need the best available science in the global fight against COVID-19.