In The Beginning Sometimes we base our treatments on false or unproved assumptions. In 1986, a young clinician who was about to embark on a long, influential career in pain medicine published a retrospective study of 38 patients who were being treated for chronic pain. The lead author, Russell Portenoy, MD, had recently completed his fellowship training in cancer pain management at the Memorial Sloan-Kettering Hospital in New York City and was curious to learn whether opioid narcotics, such as oxycodone or methadone, could be safely prescribed for the long-term management of patients with noncancer-related forms of chronic pain.1 Twenty-four patients described adequate pain relief, and 14 did not. There were few substantial gains in employment or social function that could be attributed to the opioid therapy. However, no toxicity was reported, and management became a problem in only two patients, both with a history of prior drug abuse. The authors concluded that opioid maintenance therapy was a safe option to consider when treating patients with intractable chronic pain and no prior history of drug abuse.
Within a decade of this frequently cited paper’s publication, the philosophy of pain management in patients with nonmalignant chronic pain was turned on its head. Whereas rheumatologists previously had been very skittish about using analgesics other than acetaminophen or a few selected nonsteroidal antiinflammatory drugs (NSAIDs) to manage chronic pain, we were now encouraged or even cajoled into prescribing opioids. The American Pain Society campaigned to make pain the “fifth vital sign” that doctors needed to monitor.
In the competitive world of medical regulatory agencies, the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) was not to be undone. It, too, issued new guidelines instructing hospital staff to regularly ask patients about their pain and to make treating it a priority. The now-familiar pain scale was introduced in many hospitals, with patients being asked to rate their pain from 1 to 10 and to circle a smiling or frowning face. The JCAHO published a guide that echoed Dr. Portenoy’s earlier conclusions. “Some clinicians have inaccurate and exaggerated concerns” about addiction, tolerance and risk of death. It went on to state, “this attitude prevails despite the fact there is no evidence that addiction is a significant issue when persons are given opioids for pain control”—never mind that the JCAHO booklet was sponsored by one of the major opioid manufacturers in the U.S.2
About Face
That was then, and this is now. We have learned, much to our chagrin, that opioid use in noncancer patients is fraught with a high risk of addiction. In 2010, there were 5.4 opioid-related deaths and 540 admissions for treatment of opioid addiction per 100,000 people living in the U.S. In an effort to stanch the flow of opiates, the U.S. Drug Enforcement Agency recently relegated hydrocodone, the most commonly prescribed drug in the U.S. for the past eight years, to the highly restrictive Schedule II status and moved analgesics, such as tramadol, now considered to be addictive, from a ranking similar to NSAIDs to the scrutiny of a Schedule III designation.
In a span of just 20 years, medicine has done an about face on how chronic pain should be managed. The spike in opioid prescriptions that occurred following the publication of some falsely reassuring insights helped fuel the rise of addiction disorders. Indeed, drug addiction remains a serious concern in any cohort of patients who are prescribed opiates. How did we err so badly? Small clinical trials, retrospective reviews and anecdotal case reports were blended together to generate conclusions that seemed to be inherently true, although they, too, proved to be illusory. The lack of prospective randomized studies in this field was costly, leading physicians to miss the forest for the trees.
Many examples exist of medical care being based on supposition and false assumptions. Whether it was advising extended bed rest for relieving bouts of sciatica or following an acute myocardial infarction, prescribing chronic high-dose NSAID therapy for the management of osteoarthritis or promoting the prolonged use of bisphosphonate therapy to treat low bone density, we often missed the mark. Medical progress stagnates or regresses when doctors rely on intuition or gestalt rather than solid scientific evidence to support their hypotheses.
Where Are the Data?
Physicians and their patients more readily accept decisions that are evidence based, and the paucity of such information creates voids and, hence, confusion. The lack of clear, concise data is currently clouding the highly contentious issue of medical recertification—or how doctors demonstrate to the regulatory authorities and the public that they are up to date in their training and knowledge of their specialty. Proponents of the maintenance of certification (MOC), which is required of all physicians who have gained specialty certification since 1990, maintain that this process more closely links learning goals with the delivery of better medical care and measures of greater accountability.4 MOC requires most specialists to seek recertification on a periodic basis, typically every 10 years, by successfully completing a four-part assessment designed to test their medical knowledge, clinical competence and skill in communicating with patients. The MOC program was initiated in 2000, but the pace of recertification has accelerated since 2009. Approximately 375,000 board-certified specialists and subspecialists, or about half the number the 24 specialty boards certified initially, meet MOC requirements, according to the American Board of Medical Specialties (ABMS).4
The debate has grown acrimonious, with opponents of MOC setting up online petitions urging physicians to boycott the entire process. They argue that the process is onerous, time consuming, too costly (approximately $3,000) and lacks value. For example, one activity requires candidates to document how the quality of care they provide compares with that of peers and national benchmarks. They must then apply the best evidence to improve the care they deliver with the use of follow-up assessments. Although these are worthy goals, there is currently little consensus among rheumatologists regarding what constitutes true quality care and how it can be properly measured. This whole field is still in its infancy and must still pass several hurdles before it can gain widespread acceptance.
Another concern voiced by opponents of MOC is a requirement that a secure, written examination be completed without access to outside sources of information. This condition contradicts what most medical educators stress to trainees: When practicing their craft, they should take advantage of the best sources of information rather than rely entirely on their memory. Is there any benefit to writing these exams at this stage of a physician’s career? Will it make me a better doctor? Some authors have questioned whether examination scores serve as surrogate markers rather than measures that define true clinical competency. Medicine is replete with surrogate markers that may correlate well with a particular outcome, but may not be truly causal.5
The logical belief that the current MOC process will improve a physician’s knowledge &, hence, performance, has yet to be confirmed in randomized trials.
Perhaps the most annoying aspect of the MOC is the need for a “secure” test environment. Frankly, this policy is insulting and demeaning to applicants whose identity must be confirmed by digital fingerprinting or by palm-vein recognition technology. We can board airplanes or tour the White House with far less scrutiny. Watches, wallets, pens, pencils are verboten, and only “essential medical items (e.g., nitroglycerin, asthma inhalers, diabetic supplies) may be brought into the testing room with prior approval of the ABIM.”6
The National Board of Medical Examiners (NBME) went even further. In 2007, it denied extra time breaks to a medical student, Sophie Currier, who was breastfeeding her 4-month-old daughter, stating that lactation was not covered under the Americans with Disabilities Act.7 What were they thinking? Did this exam happen to include a section on ethics and professionalism? Although the NBME initially prevailed, five years later, the Supreme Judicial Court of Massachusetts overruled the lower court decision and sided with Dr. Currier, enshrining the right of future test takers to be allotted extra time to breastfeed.
The most striking aspect in the debate over the MOC is the lack of evidence to support its current iteration. Although nonrandomized data suggest an association between initial board certification and improved clinical performance, little or no data exist demonstrating improved outcomes of care related to recertification. The logical belief that the current MOC process will improve a physician’s knowledge and, hence, performance, has yet to be confirmed in randomized trials.5
However, this has not prevented the various boards from assuming that this statement is correct. It is puzzling that this hypothesis has not been studied more vigorously. There are currently several hundred thousand participants who have completed MOC, and with an estimated 50,000 new participants each year, it would not be hard to analyze the effect of recertification on physician performance. Surely, funding to support large, randomized trials should be readily available. After all, in 2012, ABIM revenue totaled $49 million, with 62% of this income derived from certification fees and 36% from MOC fees.4
The Illusion of Validity
Psychologist Daniel Kahneman, PhD, spent much of his career rooting out characteristic flaws in human thinking. For example, he coined the term, the illusion of validity, to describe how we often reach conclusions based on false assumptions. His first recollection of this error occurred decades earlier in his life, when he served as a young Israeli Army officer. In an effort to identify recruits who demonstrated exceptional leadership skills, he served on a team of observers whose task was to witness a series of team-building exercises performed by soldiers. As Dr. Kahneman recalled:
Under the stress of the event, we felt the soldiers’ true nature would reveal itself, and we would be able to tell who would be a good leader and who would not. But the trouble was that, in fact, we could not tell. Every month or so we had a “statistics day,” during which we would get feedback from the officer-training school, indicating the accuracy of our ratings of candidates’ potential. The story was always the same: our ability to predict performance at the school was negligible.8
A similar situation may be at play with MOC. Conclusions have been reached without confirming that the assessment tools being used are measuring the appropriate data. Is the entire theory behind MOC even valid? Could an illusion of validity be responsible for leading the MOC in the wrong direction? As physicians, we are committed to developing large, randomized trials to study controversial issues. Isn’t it time for the ABMS to design, implement and fund such studies?
Simon M. Helfgott, MD, is associate professor of medicine in the Division of Rheumatology, Immunology and Allergy at Harvard Medical School in Boston.
References
- Portenoy RK, Foley KM. Chronic use of opioid analgesics in non-malignant pain: Report of 38 cases. Pain. 1986 May;25(2):171–186.
- Catan T, Evan Perez E. A pain-drug champion has second thoughts. The Wall Street Journal. Dec. 17, 2012.
- Porter J, Jick H. Addiction rare in patients treated with narcotics. N Engl J Med. 1980 Jan 10;302(2):123.
- Iglehart JK, Barron RB. Ensuring physicians’ competence—Is maintenance of certification the answer? N Engl J Med. 2012 Dec 27;367(26):2543–2549.
- Levinson W, King TE Jr, Goldman L, et al. Clinical decisions. American Board of Internal Medicine maintenance of certification program. N Engl J Med. 2010 Mar 11;362(10):948–952. http://www.nejm.org/doi/full/10.1056/NEJMclde0911205#t=cldeOpt2.
- Exam day: What to expect. ABIM website. http://www.abim.org/exam/exam-day.aspx#id.
- High court sides with former Harvard med. student in breast-feeding lawsuit. Associated Press. Apr 13, 2012. http://boston.cbslocal.com/2012/04/13/high-court-sides-with-former-harvard-med-student-in-breast-feeding-lawsuit.
- Kahneman D. Thinking, Fast and Slow. Farrar, Straus & Giroux, New York: 2011.
