Rohin Francis doesn't just question how we measure surgical skill; he dismantles the very idea that a single statistic can capture the complexity of human care. In a landscape obsessed with rankings and public scorecards, Francis argues that the metrics we trust most are often the ones driving the worst outcomes. This isn't just a critique of data—it's a warning about the hidden costs of turning medicine into a spreadsheet.
The Illusion of the Perfect Score
Francis opens with a deceptively simple scenario: two orthopedic surgeons, a married couple, and a stark difference in their reported mortality rates. One has a 1% death rate; the other, 10%. "Stats can mislead," he writes, immediately challenging the reader's instinct to crown the lower number as the winner. He reveals that the surgeon with the higher mortality rate works at a major trauma center, handling the most complex cases—road accidents, stabbings, and frail elderly patients—while the "winner" operates in a private hospital on healthy, low-risk patients. "If a surgeon with a fantastic mortality rate only achieves that by turning down cases that their colleagues would have accepted, they might be denying some patients an effective treatment." This reframing is crucial. It shifts the focus from the surgeon's performance to the system's incentives, suggesting that a "good" statistic might actually signal a refusal to help the sickest people.
The author's use of the "Kohley" couple serves as a narrative hook, but the real weight lies in his explanation of case-mix. He argues that attributing mortality solely to the surgeon ignores the dozens of other factors involved in a patient's care, from the severity of the initial injury to the hospital's resources. "Why should mortality be solely attributed to the surgeon?" he asks, a question that cuts through the noise of public reporting. Critics might note that without some form of risk-adjusted data, patients have no way to distinguish between a surgeon who avoids risk and one who simply lacks skill. Francis acknowledges this, admitting that high mortality can indicate a bad surgeon, but insists that within the normal range of variation, the data is too noisy to draw firm conclusions.
"The moment I prescribe a medication or I make an incision, I am doing the patient harm. I think we can forget this."
The Trap of Commission Bias
Francis moves from statistics to the psychology of the doctor-patient relationship, introducing the concept of "commission bias." This is the human tendency to want to do something, even when inaction is the safer choice. He illustrates this with a patient who, despite a stable condition, asks, "even if there's a 1% chance why can't we go for it?" Francis explains that while doctors are trained to fix problems, their job also requires knowing when not to intervene. "If a doctor feels that the risks of an operation outweigh the benefits, then they are subjecting a patient to an invasive procedure in the knowledge that it's more likely to harm than help." This is a profound reminder that medicine is not binary; it is a spectrum of outcomes where "success" might mean a slow decline or a life lived with severe limitations.
He traces this back to the ancient Hippocratic Oath, noting that the original text included a line about not operating on stones because the procedure caused more harm than good. "Don't do it because you will harm far more than you heal," he quotes, highlighting how modern medicine has sometimes lost sight of this fundamental principle. The author argues that nothing in medicine is binary, and the outcome is rarely a simple choice between recovery and death. Instead, there are "limitless inbetweens"—strokes, long rehab stays, or the inability to play with grandchildren. "If the latter [prolonging life] comes at the expense of the former [quality of life], then we are not thinking holistically." This section is particularly strong because it humanizes the data, reminding us that behind every statistic is a person whose life might be prolonged but not improved.
When Metrics Become the Goal
The piece takes a sharp turn into the unintended consequences of performance metrics, invoking "Goodhart's law": "anytime a metric i.e. a way to measure something becomes the goal, it ceases to be a good metric." Francis uses the famous "cobra effect" from British India, where a bounty on dead snakes led to snake farming, to illustrate how incentives can backfire. He applies this directly to the NHS, where a target of seeing patients within four hours led to hospitals discharging patients prematurely to meet the quota. "The intention was good medical care. But even though hospitals often had almost 100% compliance with the statistic, they may not actually be the hospitals offering the best care." This is a devastating critique of bureaucratic management in healthcare, showing how a well-intentioned rule can degrade the very service it aims to improve.
He shares a personal anecdote about a consultant who prided himself on discharging patients quickly, only to find that his readmission rates were the highest. "The patients he sent home ended up coming back to the hospital more often than anybody else," Francis notes. This story underscores his central thesis: that focusing on a single number distorts behavior and obscures the true quality of care. He warns that publishing individual surgeon mortality rates has created the same phenomenon, where surgeons become "wary to take on high-risk cases" to protect their stats. "Surgeons are not being selfish when they decline an operation," he argues, but the system makes it feel that way. "Having said that sometimes high mortality is indicative of a bad surgeon. So always remember there is a limited amount you can infer from a single data point."
"A good surgeon knows how to operate. A better surgeon knows when to operate, but the best surgeons know when not to operate."
The Myth of the "Top" Doctor
Francis concludes by attacking the commercialization of medical prestige, particularly the rise of "top doctor" lists in magazines and social media. He dismisses these as "meaningless phrases," often paid for by the doctors themselves. "There is no legitimate list of doctors that would make any sense," he asserts, arguing that the complexity of medicine makes such rankings impossible. He mocks the idea of a "mixed medical martial arts league" where doctors could be ranked by their ability to identify vulnerable points in the body, highlighting the absurdity of reducing clinical judgment to a competition. "When not only do such things not exist, they cannot exist. You can't rank doctors like this." This final blow to the ranking culture is both humorous and deeply serious, urging readers to be skeptical of any claim that simplifies medical expertise into a headline.
Bottom Line
Francis's argument is a masterclass in skepticism, exposing how the drive for measurable outcomes can undermine the very essence of good medicine. His strongest point is the demonstration of how metrics, when treated as goals, create perverse incentives that harm patients. The biggest vulnerability lies in the practical difficulty of communicating this nuance to a public that craves simple answers. Readers should watch for how institutions respond to these critiques, as the tension between accountability and the complexity of care is far from resolved.