Why the Need?
For a specialty in which international collaborations and a focus on outcomes seem relatively common today, why would an initiative such as OMERACT be needed? In 1990, it was a different story. At that time, outcomes measurement was a huge challenge in clinical trials, says Vibeke Strand, MD, clinical professor of medicine in the division of immunology at Stanford University in Palo Alto, Calif., and another OMERACT executive committee member. Researchers in North America used different trial endpoints from their colleagues in Europe and regulatory agencies did not accept results of trials from countries other than their own.
During post-graduate thesis work in clinical epidemiology under the supervision of Peter Tugwell, MSc, MD, at McMaster University, Maarten Boers, MSc, MD, PhD, identified 22 different outcomes measurements that had been used as endpoints in arthritis trials worldwide. (Both men would later become founding members and executive committee members of OMERACT.) This led to a key realization, explains Dr. Tugwell, professor of medicine at the University of Ottawa and the Institute of Population Health in Ontario, Canada: “How on earth were we going to come up with a recommendation for clinicians to use in clinical practice if everyone was using a different outcome?”
Dr. Boers, professor of clinical epidemiology and biostatistics at VU University Medical Centre in Amsterdam, The Netherlands, fondly recalls these early discussions with Dr. Tugwell. “The story that should go into folklore is that, as my thesis supervisor, he had made all kinds of scribbly remarks throughout my manuscript—his handwriting is famously difficult to read—and there were two remarks I could not read,” says Dr. Boers. “One was ‘Do a review on combination therapy,’ and the other was ‘Suggest a conference on outcomes measures.’ ”
Clinical researchers weren’t the only ones challenged with disparities in outcomes measures. In discussions with regulatory agencies, Drs. Tugwell and Boers found that regulators, too, had expressed frustration with the Atlantic divide. “The idea was to get a small group of people together,” Dr. Strand says. “We were trying to make sure that there would be consensus [about which outcomes measures to use] on both sides of the pond.” OMERACT 1 took place in Maastricht, The Netherlands, in 1992, and was organized around a Delphi process to achieve consensus on outcomes measures in RA. Its objectives were:
- To broaden consensus on the minimum number of outcomes measures to be included in all RA clinical trials;
- To achieve consensus on criteria for minimum clinically important improvement in patients with RA, as well as minimum important differences between treatment groups in RA clinical trials; and
- To determine whether aggregate outcomes measures or indexes are useful in assessing patients and trials.1
The 92 conference attendees filled out preconference questionnaires and then took part in discussions and voting tallies throughout the meeting proceedings. And the rest, as they say, is history.
Dr. Tugwell recalls the tensions in that first meeting while trying to reach consensus on the top seven outcomes measures. “We had invited the top people,” he says, “and asked them to ‘take off the hat’ of having developed an outcome measure—which most of them had—and, in the interest of science, be prepared to consider which outcome might be agreed to as the one everyone should use.” Preliminary voting, done on electronic keypads, reflected an almost 50-50 split over which variables should be included, recalls Dr. Tugwell. “By the end of the conference, that bimodal voting had disappeared,” he says, “and we got a unimodal agreement on the top seven outcomes—pain, tender joints, swollen joints, function, patient’s global assessment, physician’s global assessment, and acute phase reactant.”
How It Works
The initial premise for OMERACT 1 was based on this summary statement: “Clinical trials are only as credible as their endpoints.” As many articles on OMERACT have emphasized, improving endpoint outcomes measurement requires a “data-driven, iterative alignment process.”2 The procedure for developing conference agendas has evolved over the years, and topics are generated by the grassroots membership, not its leadership. OMERACT exists as a structure, supplying parameters and expertise for members to achieve a certain level of presentation.
During the earlier years, topics could be generated using the Delphi process, guided by committees until mature enough for general discussion. Now, with so many topics vying for attention, the process is a staged one: initiatives now begin as special interest groups (SIGs), when small groups of experts conduct a literature review and validation studies. The next step is a workshop, where studies are presented that help formulate and select which of the Four D domains (discomfort, disability, dollar cost, death) will constitute the focus. Finally the topic is formally presented at an OMERACT conference as a module, when evidence from the literature and from targeted studies is presented. Final selection of relevant measures is voted upon by conference attendees, and must pass the OMERACT Filter of Truth, Discrimination, and Feasibility.2 (See www.omeract.org for more information on the filter.) The process from special interest group to module can take several years, and is not for everyone, says Dr. Boers, “but if you don’t have tools to measure what’s working and what’s not working, there will be no progress, whatever drugs you develop. So we see ourselves as the slightly zany toolkit for rheumatology!”
Notable Accomplishments
OMERACT didn’t stop with reaching consensus on core sets of measures for RA. Other achievements include core sets of measures for osteoarthritis and osteoporosis, psoriasis/psoriatic arthritis, psychosocial measures, and a core set of data for cost-effectiveness evaluations. Workshops have been held for a wide range of conditions, including MRI in ankylosing spondylitis, fatigue, fibromyalgia, gout, low back pain, drug safety, and chemical biomarkers, among many others. When asked to highlight specific OMERACT accomplishments, all those interviewed praised the successful initiative of member John Kirwan, MD, professor of rheumatic diseases at the University of Bristol Rheumatology Unit (U.K.), to include patient representatives in the OMERACT process. (See “Patients as Research Partners,” above right.) “We now have 29 different groups working under the OMERACT umbrella,” Dr. Simon pointed out. “The product itself—all the supplements that have been published over the years—has been incredibly noteworthy.”
Kenneth G. Saag, MD, professor of medicine and epidemiology, and director of the Center for Education and Research on Therapeutics of Musculoskeletal Disorders at the University of Alabama at Birmingham, has served as the ACR representative since OMERACT 6. “It has now broadened to take on more structure and a growing relationship with other, longer-standing international bodies, such as the ACR and EULAR,” he says.
Dr. Saag is especially interested in watching the passion that worldwide experts bring to the OMERACT process. He points to a joint ACR/EULAR project to develop measures for gout response criteria that was seeded by the OMERACT process.
“Another huge benefit of OMERACT is that it has been a tremendous training vehicle for our junior faculty and fellows,” he says. “Junior clinical investigators can spend one-on-one time with experienced, senior-level people from throughout the world.”
Tensions and Image Issues
Inclusion of industry representatives has been a contentious issue, but OMERACT’s organizers have always strongly believed that true consensus about outcomes measures is possible only if all stakeholders are brought to the table. Dr. Saag believes inclusion of industry representatives enlivens the OMERACT process. “Therapeutic development is inherently a public-private partnership,” he says. “Having all partners at the table when we are designing measurement sets that are to be used in clinical trials leads to a richer and more relevant discussion.”
Dr. Boers suspects that some in the rheumatology community harbor an unspoken belief that “OMERACT is, in some way, an industry vehicle. We obviously feel that is not true. All you can do, of course, is just keep up the good and keep on showing that the results that come out of OMERACT are not industry results, but are scientific results.” As an example, Dr. Strand points to the successes of the newer disease-modifying antirheumatic drug trials in RA as a direct result of OMERACT’s efforts, acting with the ACR and EULAR in establishing consensus regarding outcomes measures.
Current Challenges
Executive committee members are aware that success could spoil the OMERACT process. Limiting the number of people at the meetings continues to be a challenge. OMERACT has “grown tremendously, and we’re now trying to figure out how to try to limit the number of topics at each meeting,” says Dr. Strand. The stepped plan (SIG to workshop to module) for consideration of meeting topics was one answer to the challenges of growth. After OMERACT 8, held in Malta, many attendees complained about the number of topics and the expanding meeting size.
Dr. Boers is concerned about losing flexibility in meeting organization, and he worries about mentoring new leaders for the process. “The number of participants has doubled, but the number of experts who really know how to run a workshop has not doubled,” he explains. “There is still a lack of seasoned experts to guide the process through. So, if you keep expanding, pretty soon the quality will go down.”
Another challenge: bringing OMERACT’s recommendations into the daily practice of rheumatology. Rheumatologists may recognize that use of outcomes measures is an essential part of good management, says Dr. Brooks, “but at the end of the day, very few of us actually know whether the treatments we are giving to patients are actually helping.” He urges practicing rheumatologists to learn more about the OMERACT order and to help their patients understand more about measurements and the drugs they are taking.
Summing up the OMERACT experience, Dr. Brooks says, “There are many things you do in life, and I think—and I suspect the others feel as I do—that this is something that we are all pleased about and proud of, that we’ve been a part of this movement.”
Gretchen Henkel writes the “Metrics in Rheumatology” series.
References
- Fried JB, Boers M, Baker PR. A method for achieving consensus on rheumatoid arthritis outcome measures: The OMERACT conference process. J Rheumatol. 1993;20(3): 548-551.
- Tugwell P, Boers M, Brooks P, Simon LS, Strand V, Idzerda L. OMERACT: An international initiative to improve outcome measurement in rheumatology. Trials. 2007;8:38.
- Kirwan JR, Hewlett SE, Heiberg T, et al. Incorporating the patient perspective into outcome assessment in rheumatoid arthritis—progress at OMERACT 7. J Rheumatol. 2005;32(11):2250-2256.
Patients as Research Partners
The need to include patients’ perspectives in the OMERACT research process was first acknowledged at OMERACT 5, held in Toulouse, France, in 2000, and patient representatives first attended as delegates at OMERACT 6, held in 2002 in Gold Coast, Australia. “When we had the first patient group,” recalls Dr. Brooks, “it opened our minds to looking at outcomes measures in a different way. After all, they are the people who have to do these measures and fill in the questionnaires. I don’t know of any other medical organization in the world that involves patients the way we do in measurement.” It was the participation of patients that led OMERACT to include fatigue as an outcome measure for RA.3
“Getting the patient perspective has been a neglected area,” says Dr. Saag. “We think we know what patients think is important, as we measure different aspects of their disease, but did we really ever bother to ask them?”
Pamela Richards, who lives in Bristol, U.K., and is the current chair of OMERACT’s Patient Panel, distinctly recalls her reactions when she attended her first OMERACT conference. “It was quite an amazing experience just to be with so many people who were researching the condition I had, and interested in the nitty-gritty of it,” she says. “My previous experience with doctors is that they would focus on the things they could do, but all of the other things were irrelevant. Some of the things I experienced were not in their textbooks and were not validated as being part of RA. Also, my experience of living with the condition every day had not been put to good use.” That all changed, says Ms. Richards, when she attended OMERACT 7, held in Monterey Bay, Calif.
As with many others who have participated in the OMERACT process, Ms. Richards has found stimulation to initiate other projects. “My passion is to focus on enabling other patients to become individual research partners,” she says. “Not everyone can attend OMERACT. However, people around the world can have the same kind of purpose by participating in their own area, not just as ‘token patients,’ but in a very positive way [e.g., to help structure protocols for clinical trials].”
To that end, Richards (together with Maarten de Wit from The Netherlands) was instrumental in formulating an OMERACT glossary targeted for patient representatives to familiarize them with the OMERACT process. She has also co-produced a three-hour patient-training workshop, which will debut at EULAR in June. She anticipates helping to foster patient groups such as the one connected with Dr. Kirwan’s department at Bristol, and credits her experiences with OMERACT as the catalyst for her activism. “These people are incredible academics, but they also recognize that patients are people. They see us [the patient representatives] as people they can work with, not just patients they treat.”—G.H.
