BETHESDA, MD—Drug research that includes information about benefits and harms to specific types of patients would be a boon for rheumatologists frustrated with a limited number of expensive drugs plagued by myriad side effects and a dearth of data comparing how well they work in particular groups of patients.
That situation may be changing, thanks to recent action by the federal government and insurers. The most recent reauthorization of the Prescription Drug User Fee Act (PDUFA V) places greater emphasis on risk–benefit analysis, while the federal healthcare reform law funds comparative effectiveness research (CER). Meanwhile, insurers are increasingly using health technology assessment (HTA), which emphasizes attention to how drugs perform in certain patient subgroups, said speakers here at the DIA/FDA Statistics Forum 2013, held April 21–May 1.
At the conference, statistics experts from academia, government, and drug companies gave advice about designing better drug trials, improving measurement of interventions’ benefits and harms, and getting information about different types of patients in intended treatment populations.
Evaluating Benefit and Risk
PDUFA V helpfully stipulates that the Food and Drug Administration’s (FDA’s) assessment of drugs should include quantitative consideration, plus subjective qualitative weighing of evidence. But PDUFA V doesn’t specify a standard methodology for doing so, leaving it up to researchers to best conduct risk–benefit analyses and evaluate overall patient utility, said Scott Evans, PhD, senior research scientist Harvard School of Public Health in Boston.
Neither does PDUFA V address certain limitations of traditional analyses. Separating marginal analyses of benefits and harms, as researchers currently do, won’t distinguish between two important, different scenarios, he said. These are benefits occurring in the same patients that experience toxicity, and benefits occurring in different patients than those that experience toxicity. In the first case, the issue of weighing benefits and harms is important. In the second, subgroup identification and analysis is key. “To distinguish these two scenarios, within-patient analyses are needed, whereby efficacy and safety data are combined within-patient. Then results are summarized by intervention and compared. Such analyses are more consistent with the evaluation of individual patient care,” Dr. Evans explained.
He suggested that the association between benefits and harms and heterogeneity of effects be considered part of standard evaluation of drugs.
Such an approach may have yielded additional information from the BEST (Beta-Blocker Evaluation of Survival Trial) study. Results published in the New England Journal of Medicine (NEJM) in 2001 showed that bucindolol resulted in no significant overall survival benefit in a demographically diverse group of 2,708 patients with New York Heart Association class III and IV heart failure.1 Dr. Evans suggested a composite ordinal variable derived from several benefits and harms—such as hospitalizations and cardiac events—may provide a more complete characterization of disease burden and help to elucidate treatment effects.
Researchers often use a composite event-time endpoint—such as time to the first of death, hospitalization, or cardiovascular event—in cardiovascular trials, but such analyses suffer from the complexities of competing risks and components with varying importance, he added. Using an ordinal categorical variable based on benefits and harms might have avoided complexities associated with competing risk from death and informative censoring in the NEJM paper, he said.
Dr. Evans also presented a multidimensional approach to trial design, monitoring, and analyses that involves thinking about efficacy and safety as coprimary endpoints. For example, a trial may be designed to evaluate noninferiority with respect to efficacy and superiority with respect to safety, or vise versa, he suggested.
Better Risk–Benefit Analysis
Rheumatologists and other physicians often find it difficult to discern which drugs are safest for their patients. That’s because published drug studies measure harm quite differently, said Steven M. Snapinn, PhD, vice president for Global Biostatistical Science for Amgen, Inc. Some look at absolute and relative measures of harm, others look at differences relative to controls, while others don’t have controls. “Is there a right approach? Can you tell which harm is of a larger magnitude?” he asked.
Comparisons of benefit and harms should involve absolute measures and account for follow-up duration, he suggested. But when a study doesn’t consider time, it could look at event rate among treatment and control arms, and determine the increase in absolute risk, he suggested.
While this approach measures public health impact, it presents challenges. The approach can minimize the magnitude of harm and makes taking into account follow-up duration challenging, especially if hazard rates aren’t constant, or proportionate over time in the two arms, he added.
Assessing the clinical meaningfulness of benefits and degree of harm is also key, Dr. Snapinn maintained. He identified hazard ratio as the most appropriate measure for assessing clinical meaningfulness of benefit and absolute risk increase as a consistent measure for harm. For risk–benefit assessment, Dr. Snapinn recommended comparison of absolute risk reduction, or efficacy, to determine benefit and increase of such risk to determine harm.
Assessments must be based on rates, not proportions, while interpretation of study data should incorporate judgment regarding relative importance of efficacy and safety endpoints, Dr. Snapinn added.
He noted, however, that this model carries some limitations. It doesn’t account for competing risks, and results may not hold for all survival distributions.
Comparative Effectiveness Research
Meanwhile, the Affordable Care Act emphasizes a focus on determining how well drugs work for particular types of patients through CER, which assesses the benefits and harms of interventions, highlighting comparisons and outcomes that matter to patients; considers individual preferences and autonomy; and includes a wide variety of settings and diverse participants. CER addresses burden to individuals and availability of services, technology, and personnel.
“The nation has made a commitment to CER,” Sally Morton, PhD, professor and chair of biostatistics at the University of Pittsburgh. Dr. Morton noted that law established the Patient-Centered Outcomes Research Institute (PCORI) as a source of CER funding. “The national research agenda places the interests of patients and the community at the forefront,” she said.
CER involves many steps, each with its own challenges, said Dr. Morton. The first, systematic review is “the foundation for CER,” she said. Systematic reviews identify, select, assess, and synthesize the findings of similar but separate studies and can help clarify what is known and not known about the potential benefits and harms of drugs. It can be difficult to choose among several existing different standards for conducting systematic reviews, she said.
Systematic reviews must assess both benefits and harms of interventions, said Dr. Morton, adding that evaluation of harms can include consideration of both randomized clinical trials and observational studies. Institute of Medicine standards state that observational studies should complement, rather than serve as a substitute for, randomized studies, but finding high-quality observational studies and synthesizing them can be difficult, she added.
Other challenges include a lack of direct evidence for comparing benefits and harms of treatments and difficulties inherent in evaluating the strength of the body of evidence for treatments. Such evaluations are imperfect processes, Dr. Morton said, recalling a study in which she asked pairs of reviewers to grade evidence from randomized controlled trials and observational studies on their own, and reach a consensus. “Even experienced reviewers didn’t reach the same conclusions on the strength of evidence,” she noted. “For CER, this highlights the importance of communication and transparency that meets the needs of all stakeholders.”
HTA’s Impact Grows
HTA—which involves risk–benefit analysis in determining if government-funded health services are safe, efficacious, and cost effective—is becoming more important in drug development, said Richard Willke, PhD, acting head of the Global Market Access Primary Care Business Unit at Pfizer, Inc.
Noting that the U.S. lags behind Europe, Canada, and other countries in use of HTA, private U.S. insurers and, to a certain extent, Medicare, now use it, too. For example, Wellpoint includes CER and observational data in its outcomes-based formulary process, and looks at “how drugs perform in the real world” by conducting internal analyses of claim data, Dr. Willke said.
Medicare uses HTA in National Coverage Determinations about controversial or expensive interventions, primarily for outpatient drug treatment covered by Part B. Dr. Willke cited a 2012 study of 195 such determinations made between 1999 and 2007. It found positive Medicare national coverage decisions were associated with fair to good quality supporting evidence, lack of an alternative intervention, and evidence of cost effectiveness.2
Looking at Particular Patients
In Europe, such decisions are made with more focus on different sorts of patients. In Britain, the National Institute for Clinical Excellence (NICE) is well known for its consideration of patient subgroup data. NICE coverage decisions about particular treatments are based on cost effectiveness in specific patient subgroups and are not always the same for all subgroups, Dr. Willke said.
Understanding the characteristics of intended treatment populations that are of most interest to payers can aid in strategic drug product development and commercial planning, he added. Researchers should look at patient populations and determine who is undertreated or unresponsive to treatment, expensive to treat, adherent or not, and at high risk for progression of disease or acute events, Dr. Willke suggested. He also recommended that researchers identify key comparators, evidence gaps, and likely effects of trial results, and then parse out different types of patients before identifying an intended treatment population.
“Payers want trials to reflect clinical practice, and to show which patients will benefit,” said, Christine Fletcher, executive director of biostatistics at Amgen, Inc., who recently set up an HTA biostatistics group at her company. She noted that Germany has a new law requiring trials to investigate which patients get the most benefits from a drug. She gave advice about how statisticians can design drug trials that incorporate payer requirements so that marketed products ultimately receive insurer reimbursement.
To demonstrate a drug is cost effective, it is important to show how a drug performs relative to other interventions in subgroups of patients of interest to insurers, she said. Many studies of relative effectiveness include indirect comparisons, also known as network meta-analyses, which are useful for drugs that have not been studied together in head-to-head clinical trials. Indirect comparisons require a number of assumptions and careful interpretation of results due to potential biases and limitations of the methodology, said Fletcher, who recommended a few ways to add statistical value to indirect comparisons.
These include understanding the existing clinical evidence, assessing clinical and statistical sources of heterogeneity, investigating heterogeneity and variability in treatment effects, and conducting meta-regression analyses to explore important prognostic variables. Key to indirect comparisons is conducting extensive sensitivity analyses that enable the robustness of conclusions to be evaluated, she added.
These tactics make drug products more marketable. “Understand that the traits of the intended treatment population are of the most interest to payers and can aid in strategic development,” Willke said. Such a focus would be good news for rheumatology patients as well.
Deborah Levenson is a medical writer based in College Park, Md.
References
- Beta-Blocker Evaluation of Survival Trial Investigators. A trial of the beta-blocker bucindolol in patients with advanced chronic heart failure. N Engl J Med. 2001;344:1659-1667.
- Chambers JD, Morris S, Neumann PJ, Buxton MJ. Factors predicting Medicare national coverage: An empirical analysis. Med Care. 2012;50: 249-256.
