The Why & What of the ACR’s Clinical Practice Guidelines

Dr. Caplan

Dr. Caplan explains that the GRADE format helps the guideline developers avoid vague concepts while shaping specific recommendations around choices that clinicians address. “The PICO questions articulate medical scenarios in a very structured format, and then evidence is gathered around that scenario.” PICO questions are one evidence-based model used to help frame clear questions. PICO questions must specify the Patient population addressed, the Intervention, the Comparison and the Outcome(s) being examined in a specific clinical situation.

Each PICO-based recommendation receives a strong or a conditional recommendation for or against a specific intervention, as determined by the voting panel. Each recommendation also receives a level of evidence rating: very low, low, medium or high.

Four factors influence the strength of a recommendation under the GRADE method. These are the quality of the evidence, the balance between intervention benefits and risks/harms, cost and variability in patient values and preferences. A weakness in one of these areas can still result in a strong recommendation if the other criteria are convincingly robust. Conversely, an intervention that is particularly impressive in one area may receive only a conditional recommendation if other factors make it less desirable.

Example: For patients with established rheumatoid arthritis and low disease activity who have never taken a disease-modifying anti-rheumatic drug (DMARD), the 2015 GRADE-based rheumatoid arthritis guidelines give a strong recommendation for DMARD therapy (preferably methotrexate) over a tumor necrosis factor (TNF) inhibitor. However, the evidence is of relatively low quality. The justification for this strong recommendation is that DMARD therapy is less expensive and has an extensive safety record with well-documented efficacy in clinical experience.³

Dr. Neogi provides a different type of example, “If something is astronomically expensive, and the benefits are only incremental above other more cost-effective treatments, you wouldn’t give that one a strong recommendation as first-line therapy, even if the randomized controlled trial showed that it was superior to placebo.”

Each recommendation also receives an evaluation in terms of the evidence quality, based on the available data. Using specific software tailored to GRADE assessment, reviewers use the GRADE criteria to assess the quality of the available evidence. This grading incor­porates a number of different characteristics, including the number and specific types (randomized controlled trials, observational, etc.) of studies available, risk of bias in included studies, inconsistency of study results, indirectness of evidence and imprecision.

In other words, GRADE assessment not only considers the trial type but also the level of quality of those studies in relation to the clinical question being asked in the guideline. As Dr. Caplan explains, “In GRADE, a really well done observational study could be increased in terms of its value. In contrast, a poorly developed randomized controlled trial with major potential for bias could be downgraded in its quality.” In addition, a high-quality study might be downgraded slightly in the guideline evidence report because the questions it addresses aren’t exactly the same as those in the guideline.