Introducing the CDC Tier-Classified Guidelines Database

Over the last several years, OPHG has hosted a “Tier Table” database of genomic applications (i.e., clinical scenarios involving genomic testing) sorted into one of three tiers using a method described in Clinical Pharmacology and Therapeutics, 2014. The purpose of creating the database was to help organize evidence (such as recommendation statements contained in guidelines, systematic reviews, and policy determinations). Tier codes provided a rough indication of evidence quality (for instance whether or not guideline statements were based on systematic review) as well as the nature of the recommendation (whether a recommendation statement was for use of a test, against use, or neutral). So, for example, genetic testing in a scenario recommended in a guideline based on systematic review would be coded as Tier 1, suggesting that there is some reasonable evidence to support implementation. A scenario coded Tier 2 could arise, for example, from guidelines, regardless of whether or not they were based on a systematic review, that found insufficient evidence to recommend for or against use of a test. So Tier 2 applications could be considered as not ready for routine implementation, based on existing evidence, but potentially useful in decision making. Tier 3 applications could either arise when there was no relevant synthesized evidence (guidelines, systematic reviews, formal technology assessments, etc.), OR when there was an evidence source recommending against use of a test in a given scenario. Tier 3 clinical scenarios may be interpreted as unsuitable for implementation in clinical or public health practice, based on current evidence. Presenting the results of this type of coding in our Tier Table database was intended to provide a limited set of examples on how we applied our criteria, while encouraging others to develop similar schemes using criteria most suitable for their own purposes Stakeholders have often let us know that a more comprehensive database would be desirable. This is a reasonable request, since the overarching goal of tiered classification has been to organize our weekly horizon scanning results (well over 20,000 articles collected in the last seven years). While not all articles in that bucket would be classifiable by tier, many would be, and unfortunately our process has not proven quick and simple enough to handle this load, given resource requirements. In this blog we introduce a modified approach (illustrated in the new Tier-Classified Guidelines Database) that could prove more efficient than the previous method. To make the process more nimble, we are making a switch from tier coding individual clinical scenarios contained within guidelines, to coding the actual guideline documents themselves according to the highest tier level clinical scenario(s) contained within.

A key bottleneck in our previous attempt to assign tier levels has been the extraction and presentation of clinical scenarios. As we noted in our 2014 paper, “[e]ven when clinical scenarios and recommendations are clear, some degree of interpretation, condensation, and/or simplification is necessary.” There may often be uncertainty as to whether the original intent of a guideline has been preserved through the process, since guideline development groups generally try to be very careful about the language they use, and make their writing as precise as possible. In addition to being the rate limiting step of the tier classification process, identifying and presenting clinical scenarios by tier level may also deter readers from opening and reading the actual guidelines, if they feel they can get the information they need more quickly from our table. Moreover, most guidelines are highly complex and nuanced documents that attempt to present all information in a nuanced way. For these reasons, we have decided to try out an approach where we assign tier levels to the guideline documents themselves, rather than to summaries of the clinical scenarios they describe.

As mentioned above, results from our latest approach are being cataloged in the Tier-Classified Guidelines Database. The criteria for assigning tier levels are identical to those we have used previously to sort clinical scenarios, but the present approach involves sorting guideline documents (and not clinical scenarios) according to the highest tier level that can be assigned to a clinical scenario they contain. While this still involves identifying clinical scenarios in the guidelines, and even deciding what tier level criteria these scenarios would meet, we no longer extract and summarize the scenarios. Let’s consider a hypothetical example. We identify a guideline document among horizon scanning results, and look at the abstract to see if there is a clinical scenario involving genetic testing. If there is not one, we consider the tiered classification scheme to not be applicable, and the guideline is not included in the Tier-Classified Guidelines Database. If there is one or more such scenarios, we look for those that would satisfy Tier 1 criteria (for guidelines other than Centers for Medicare & Medicaid Services coverage determinations, or U.S. Food & Drug Administration decisions, this boils down to the question of whether or not the guideline was based on a systematic review). If any of the clinical scenarios in the guideline meet Tier 1 criteria, then the guideline is coded Tier 1. If there are no Tier 1 clinical scenarios but there is at least one meeting Tier 2 criteria, then the guideline is coded Tier 2. Finally, if there are no Tier 1 or Tier 2 clinical scenarios, but one or more meeting Tier 3 criteria, then the guideline is coded Tier 3. There are some important caveats that go along with this approach, and we have summarized these in an FAQ section, which is reprinted below in a separate blog (link). While clinical scenarios sorted by tier level are not presented, we hope that being able to quickly identify a bolus of guidelines that contain, for example, one or more Tier 1 clinical scenarios will be useful. Readers will need to identify and summarize the clinical scenarios for themselves, by reading the original guideline documents for themselves, but we believe that this is the best way for people to understand the state of the evidence anyway. As always, we welcome your questions and comments.