I. The woman in Frankfurt

On November 25, 1901, a 51-year-old woman named Auguste Deter was admitted to the municipal asylum in Frankfurt. She was disoriented and suspicious, and she could not hold a thought long enough to finish it. The physician who examined her, a young psychiatrist named Alois Alzheimer, asked her to write her name. She began and could not finish. “I have lost myself,” she said as she tried. Her case file, missing for most of the twentieth century, was rediscovered in a Frankfurt hospital archive in 1995.

Auguste Deter died in April 1906. Alzheimer, by then working in Munich in the laboratory of Emil Kraepelin, the most influential German psychiatrist of his time, had her brain sent to him. Under the microscope he found what would become the two signatures of the disease: plaques between the neurons and tangled fibrils inside them. That November he presented the case at a meeting of psychiatrists in Tübingen. No one asked a question and the local press gave the lecture two lines.

Four years later, Kraepelin gave the condition a name. In the eighth edition of his textbook, he called it Alzheimer’s disease. Historians still debate why he declared a new disease so quickly on the strength of a few cases. The scope of the label, however, was clear. It described a rare dementia of middle age, separate from the ordinary senility of old age.

For more than half a century, that is what the term meant. Then, in April 1976, Robert Katzman, a neurologist at Albert Einstein College of Medicine, published a two-page editorial in Archives of Neurology arguing that the distinction made no sense. The plaques and tangles of older patients looked like those of younger ones. Senile dementia, he concluded, was Alzheimer’s disease. With that reclassification, a rare condition became, by his estimate, the fourth or fifth leading cause of death in the United States. Federal funding for Alzheimer’s research grew from about $5 million in 1980 to $647 million in 2005. After Congress added $100 million for fiscal year 2026, annual NIH funding for Alzheimer’s and related dementias passed $3.9 billion.

Katzman had good reasons to want dementia in older people taken seriously. But it’s worth underscoring what happened. Nothing in the biology changed in 1976. The label changed, and the label then decided what everyone would look for. A category drawn around one woman’s brain, named in one man’s textbook and enlarged by another man’s editorial became a single disease with a presumed single cause. Nearly everything since has been studied inside it.

A label is a human judgment that hardens into infrastructure. It sets the inclusion criteria for trials, the endpoints regulators accept, the codes on death certificates and the categories in every dataset built afterward. Some of the questions it frames lead somewhere. Many lead into dead ends that take decades to recognize, because the label itself is rarely what gets tested.

II. Heroes and villains

This week, Nature ran a news feature by the science journalist Alison Abbott on people who resist dementia. It’s careful reporting about serious science. Henne Holstege, a geneticist at Amsterdam University Medical Center, spent five years having grants rejected before she could begin studying the very old who stay cognitively intact. Her field has since identified a dozen or so rare variants linked to resilience, with evidence that varies widely in strength. Several sit in APOE, the gene that carries the strongest common genetic risk for late-onset Alzheimer’s. One, found in an Icelandic cohort in 2012, sits in the gene for the amyloid precursor protein. Another, in the gene for fibronectin 1, points to the walls of the brain’s blood vessels.

If we read the piece for the vocabulary it uses, a story appears underneath the science. The resilient variants seem to possess superpowers. The brain has defense mechanisms against toxic proteins. One allele dials risk up and another dials it down. Variants confer resistance, protect us and do so safely. Near the end, Laura Nisenbaum of the Alzheimer’s Drug Discovery Foundation describes a rare variant as “nature showing us a drug target that already works in a human being.”

In genetics, “protective” has a narrow technical meaning: a variant is associated with lower odds of a defined outcome in a defined population. As shorthand, it is defensible. But scientific shorthand compresses complex phenomena, and the compression hides its assumptions that can be arbitrary or misguided. This one assumes that there is actually a disease, that it has a cause, that the cause is known, and that each variant stands either with us or against us. Each of those is a claim about biology. None has been settled, and the last is not the kind of claim biology can settle. APOE4 shows us why.

III. The ancestral allele

APOE4 is the variant the Nature article (and many others) cast as the villain. In a 1997 meta-analysis of 5,930 patients and 8,607 controls, Lindsay Farrer of Boston University and colleagues found that among people of European ancestry, one copy of APOE4 raised the odds of Alzheimer’s about threefold relative to the most common genotype, and two copies raised them far more. Yet APOE4 is not a recent “error.” Sequence analysis led by Stephanie Fullerton, then at Pennsylvania State University, indicates that APOE4 is in fact the ancestral form of the gene. APOE3 and APOE2 are derived from it, and APOE3 spread only over the past 200,000 years. For most of human evolution, APOE4 was the norm.

In 2017, a team led by Ben Trumble, an anthropologist at Arizona State University, studied 372 Tsimane, forager-horticulturalists in the Bolivian Amazon who live with heavy parasite burdens. Among people without APOE4, a high parasite burden went with poorer cognitive performance. Among older APOE4 carriers with the same burden, cognition held steady or slightly improved. The allele that predicts decline in Boston appeared to preserve cognition in a river village where infection is constant.

Late-onset dementia usually appears after the reproductive years, when natural selection has little left to act on. Michael Gurven, an anthropologist at the University of California, Santa Barbara, and a coauthor of the Tsimane study, has noted that an allele can persist despite causing harm late in life, particularly if it confers some advantage earlier. To natural selection, APOE4’s late-life effects were close to invisible.

APOE4, then, is neither a risk gene nor a protective one. Its effects depend on the environment it meets, the age at which it is measured and the outcome we choose to count, and the label “risk allele” folds all three into a verdict that mentions none of them. The Nature piece offers its own reminder: W276C, one of the protective variants it describes, has so far appeared only in a cohort of Native American ancestry, which suggests that a variant’s meaning travels with the population and setting in which it was found.

IV. The anomalies

The Nature article frames Alzheimer’s the standard way: amyloid clumps into plaques, the plaques set off inflammation and tau tangles, and the brain degenerates. If plaques start the process, people with plaques should eventually develop dementia. Many do not. A 2015 meta-analysis in JAMA of 55 studies, led by Willemijn Jansen and Pieter Jelle Visser in the Netherlands, found that amyloid positivity among cognitively normal people rose from 10% at age 50 to 44% at age 90.

The most famous exception is a woman from Antioquia, Colombia. She belongs to an extended family that carries PSEN1 E280A, a mutation that drives early-onset Alzheimer’s with near certainty. Carriers typically show cognitive impairment around age 44 and dementia around 49. She stayed cognitively intact into her seventies. In 2019, a team led by Joseph Arboleda-Velasquez of Mass Eye and Ear, Yakeel Quiroz of Massachusetts General Hospital and Francisco Lopera of the University of Antioquia reported in Nature Medicine that she carried two copies of the APOE3 Christchurch variant. Her brain carried an unusually heavy amyloid burden. Her tau burden was low, and her brain metabolism looked healthy.

Inside the amyloid frame, she was protected from the consequences of her plaques. Outside it, she is evidence that plaques were not sufficient to produce her disease, and that whatever tracked her cognition lay somewhere else.

Even this anomaly deserves the scrutiny we would give any hypothesis. In August, an investigation in Science by the reporter Charles Piller raised concerns about how the research team identified its protective variants, including changes to patient data and apparently altered or duplicated images in several papers. Those questions are not settled, and laboratory work from other groups still suggests the Christchurch variant changes how ApoE behaves. But if the concerns hold up, some of the field’s most celebrated protective variants may prove to be partly products of the story researchers wanted to tell. The label cuts both ways. “Protective” can seduce as easily as “risk” can condemn.

The drugs tell a similar story. Lecanemab and donanemab remove plaque with remarkable efficiency. In the CLARITY AD trial, lecanemab slowed decline on the CDR-SB, an 18-point clinical dementia scale, by 27% over 18 months, a difference of 0.45 points. In TRAILBLAZER-ALZ 2, donanemab slowed decline by 35% on its primary measure in patients with low to intermediate tau. Both have a risk of brain swelling and microbleeds, and whether patients and families can feel the benefit remains in dispute. Then, in June 2024, Nature retracted a 2006 paper by Sylvain Lesné, Karen Ashe and colleagues at the University of Minnesota that had tied a specific amyloid species, Aβ*56, to memory loss. The paper had been cited more than 2,000 times, and Ashe, its senior author, acknowledged that figures in it had been manipulated. Lesné disagreed with the retraction. Defenders of the amyloid hypothesis are right that the retraction did not refute it but it did show how much weight the field had placed on a single result.

Thomas Kuhn, the physicist turned historian of science, argued in The Structure of Scientific Revolutions that scientists rarely abandon a paradigm when they meet an anomaly. They adjust the theory or set the anomaly aside until the exceptions become too many to manage. The resilience literature does something quieter. It names each exception: cognitive reserve, a protective variant, a super-ager. The anomaly becomes a property of the person rather than a problem for the theory. Resilience is what we call an anomaly once we have decided it belongs to the patient and not to the model. It’s a convenient way of protecting our biases.

V. Junk, drivers and other verdicts

Biology has paid for this habit before. In 1972, Susumu Ohno, a geneticist at City of Hope in California, published a paper titled “So much ‘junk’ DNA in our genome.” Ohno meant something specific, and historians of genetics still argue about how far later readers stretched it. The word did its own work anyway. Wojciech Makalowski, a computational genomicist then at Pennsylvania State University, later wrote that the term kept mainstream researchers away from noncoding DNA for years. Regulatory elements, noncoding RNAs and transposons that shape how genes are expressed spent part of that time in a bin labeled trash.

In oncology, we have a more recent version. In 2018, Iñigo Martincorena and Phil Jones of the Wellcome Sanger Institute sequenced normal esophagus from nine organ donors aged 20 to 75. By middle age, clones carrying mutations in cancer genes had colonized much of the tissue. NOTCH1 mutations were present in 12% to 80% of cells, several times more common in normal esophagus than in esophageal cancer. A mutation filed as a cancer driver was more at home in healthy tissue than in tumors.

Three years later, Bartomeu Colom, Jones and colleagues showed in mice that mutant clones in normal epithelium outcompete and eliminate most emerging tumors. Whether a nascent tumor survived depended not only on its own mutations but on the fitness of its neighbors.

“Driver” was a verdict reached by looking only at tumors. Once the tissue was described as it is, a rule became visible. Clones compete for space, selection depends on the neighborhood, and the same mutation can expand a tumor in one setting and crowd one out in another. That rule says more about cancer than the verdict ever did.

VI. The universe is indifferent

Each of these labels assigns a value before the description is finished. APOE4 is a risk. Christchurch is protection. Noncoding DNA is junk. NOTCH1 is a driver. The plaque is toxic. Each verdict sounds like a finding, and each one narrows what a researcher thinks is worth examining next.

But nature makes no such assignments. An allele does not harm or protect. A plaque does not attack. A clone doesn’t betray the tissue it came from. Each follows the same rules of chemistry, selection, competition and energy that govern every other process in the body. Nature is indifferent in every direction. It has no preference for our survival and none against it.

The French philosopher and physician Georges Canguilhem worked through a version of this problem in The Normal and the Pathological, which began as his medical thesis in 1943. He argued that disease is not a foreign entity invading a healthy body but another way of living, one in which the organism tolerates a narrower range of conditions. He also argued that “normal” and “pathological” are not statistical facts. They are values, rooted in the living being’s relation to its environment. Whatever one makes of the rest of his philosophy, the core point holds. The molecules do not know which side of that line they are on.

None of this makes the judgment illegitimate. Auguste Deter lost herself, and the loss was real. But the judgment is ours, and it should come after the description, not before it. A label is the point at which human preference enters the science, usually hidden from view.

Research that starts from placing human value judements on biological phenomena looks for culprits and saviors. Research that starts from first principles and neutral mechanistic characterization looks for rules. Culprits and saviors belong to a particular story with a begining, middle, and end. Rules generalize and are eternal.

VII. The inheritance

Labels have narrowed what biology could see for more than a century. Machine learning threatens to make that narrowing propogate into the future with great efficiency.

The most comfortable form of machine learning in biomedicine is typically supervised. A model learns to reproduce the categories it is given, and its performance is measured against those same categories. Train a classifier on brain scans labeled “Alzheimer’s” and it will learn whatever best separates them from the rest, which, given how the label has been defined, is largely amyloid. By every metric we choose, it will be accurate. It cannot discover that the category is wrong, because the category is its ground truth. Whatever does not fit the training distribution registers as error.

In 2018, Thomas Stoeger, Luís Amaral and colleagues at Northwestern University showed that the attention a human gene receives is predicted less by its importance to physiology than by chemical and biological features that made it easy to study in the 1980s and 1990s. Fewer than 10% of human genes are the subject of more than 90% of published papers. Gene annotations, pathway maps and variant classifications are built from that literature, and they are now training data. A model trained on the published record absorbs its frame along with its facts: the amyloid cascade, the grammar of risk and protection, the genes no one examined.

Ask your favorite LLM about resilience and resistance in Alzheimer’s disease, and it will retell, fluently and at scale, the story we’ve been telling ourselves.

Labels embedded in infrastructure also outlive the people who made them. Kraepelin’s category stood for 66 years before it was redrawn, and when it was, it was enlarged rather than questioned. Benchmarks, ontologies and the models trained on them will shape what the next generation of scientists regards as settled. If the first generation of biological AI is built on our verdicts, it will reproduce them with more confidence and reach than any textbook, and the chance to use these systems to see biology on its own terms could be lost for a generation.

The alternative already exists. In 2021, Alexander Rives and colleagues at Facebook AI Research trained a language model on 250 million protein sequences with no labels at all. Information about secondary and tertiary structure emerged in its internal representations, learned from the statistics of evolution alone. The same year, Jonathan Frazer, Debora Marks and colleagues at Harvard Medical School and the University of Oxford introduced EVE, a generative model that predicts whether human protein variants cause disease using only the variation of those proteins across species. The authors described clinical labels as “sparse, biased and of variable quality.” EVE, trained without them, outperformed methods trained on them.

Neither model is free of human choices. People selected the training sequences, designed the architectures and chose the benchmarks, and EVE’s predictions were ultimately validated against the same clinical labels its authors set out to avoid. The evolutionary record has its own limits. A sequence conserved across species reflects what preserved fitness in past environments, not what helps or harms in a present one, which is the lesson of APOE4. What sets these models apart is where they start. Because they learn from what nature recorded rather than from what we concluded, they can find structure we never labeled. ESM recovered features of protein structure it was never shown, and EVE produced evidence to classify more than 256,000 human variants that clinical genetics had left as variants of unknown significance.

VIII. Characterize and describe first

Putting description first would change at least four things in practice.

The first is the unit of study. “Alzheimer’s disease” is a clinical and pathological label that likely spans several processes converging on similar symptoms: lipid transport, protein aggregation, immune activation, vascular integrity and energy metabolism. Thoracic oncology has already been through this. Non-small cell lung cancer was once a single diagnosis treated with the same chemotherapy backbone. Described at the level of mechanism, through EGFR, ALK, ROS1, KRAS and the rest, it broke apart into many diseases, and outcomes changed. The label had been hiding the biology all along. Dementia has not yet had that moment.

The second is what gets measured. A plaque on a scan is a snapshot of a state. The rules live in how a system moves between states over time. Longitudinal, multimodal data, combining imaging, fluid biomarkers, genomics and clinical course and modeled as dynamics, would let the resilient and the declining be described by the same equations, instead of being sorted into heroes and casualties.

The third is what we ask of our data and our models. A variant’s effect should be reported along with the conditions under which it was observed: the environment, the age of the carriers, their ancestry and the outcome that was measured. Models deserve the same discipline. Reproducing our labels is a low bar. A more useful test is whether a model trained on raw imaging, sequence and longitudinal clinical data, before any diagnosis is attached, finds structure we did not put there, such as subgroups of patients, trajectories or mechanisms that cut across the categories we use. Self-supervised learning makes that possible, and our benchmarks should reward it.

The fourth is where judgment enters. It belongs at the point of intervention, and there the logic of the Nature piece has a real precedent. In 2006, Jonathan Cohen and Helen Hobbs of the University of Texas Southwestern Medical Center reported that Black participants carrying nonsense mutations in PCSK9 had 28% lower LDL cholesterol and an 88% lower risk of coronary heart disease over 15 years. Within a decade, PCSK9 inhibitors were approved. That worked because the mechanism had been described before anyone decided what to do about it: less PCSK9, more LDL receptors, less circulating LDL. Dementia is not yet described that well. A variant that acts from conception, in a particular genetic background, across eighty years, is a different experiment from a drug started at seventy.

Auguste Deter told her physician she had lost herself. More than a century later, we still describe what happened to her in the language of loss, risk and protection. Her brain knew none of those words. It followed rules we have not yet written down.

Life is not good or bad. It simply is, and its rules are indifferent to our preferences. That indifference is not an obstacle to overcome. It’s what makes the rules discoverable. Every label we attach is a hypothesis presented as a fact. We’re now building machines that can read biology at a scale no person can, and whether they find the rules or simply repeat our verdicts depends on what we teach them to see first. Be wary of labels. Be warier still of teaching them.

Subscribe now

At four in the morning, two weeks into a pain in his upper back that would not go away, Sid Sijbrandij went to an emergency department. He had first noticed the pain while bench pressing, and it had come and gone before. The physicians reassured him that there is no such thing as a two-week heart attack and sent him home. A few hours later his own doctor called with an unexpected question: did he know how to meditate? His blood pressure at the hospital had been 170. The pain might be his aorta beginning to tear. He should bring his pressure down and go back immediately.

It was not his aorta. It was a six-centimeter osteosarcoma growing out of his T5 vertebra, a rare bone cancer in a 45-year-old man who, the year before, had taken his software company, GitLab, public.

He received excellent care at a major hospital. But when standard of care ran out, he sought to learn as much as he could about his cancer to open up new options. His team jumped through substantial hoops to get the raw data behind the sequencing tests that had been ordered by his physicians and analyzed the genomics of the tumor. This identified vulnerabilities that were not called out in the standard report.

The difficulty was not simply obtaining the data, but looking beyond what the standard report had identified as actionable. That distinction exposes the limits of a principle most of us in medicine learned in training: order only what you can act on. The rule made sense when the set of possible actions was small and fixed. It becomes a constraint when what we can learn from a patient’s tumor (and do in response) outpaces the guidelines meant to govern care. Guidelines have spared many patients from guesswork, and teaching physicians to follow them was one of medicine’s real achievements. The next frontier, however, is teaching them to follow the patient.

That shift begins with seeing what guidelines rest on. A test is worth ordering if it points to a proven treatment, and a treatment is proven if it works in a trial. But trials enroll narrowly selected patients, often younger and healthier than those who will later receive the drug, and they report what happened to the group rather than to any one person in it. The patient in the clinic rarely resembles the patients in the trial, and even the patients in the trial do not resemble its average. For more than sixty years, medicine has asked a simple question: does this work? What it has actually meant is much narrower: does this work on average? And the average, it turns out, is the least precise estimate of what will happen to any one patient.

I. The myth of the average man

American drug law was written in the aftermath of tragedy. After a sulfanilamide elixir dissolved in diethylene glycol killed more than one hundred people in 1937, the Federal Food, Drug, and Cosmetic Act of 1938 required manufacturers to show that a new drug was safe. After thalidomide, which Frances Kelsey kept off the American market, the Kefauver-Harris Amendments of 1962 added a second requirement: substantial evidence of effectiveness from adequate and well-controlled investigations. Is it safe, and does it work. The randomized controlled trial became the instrument for answering the second question, and it has served patients extraordinarily well.

The second question, however, carries an assumption inherited from the nineteenth century. In 1835, the Belgian astronomer Adolphe Quetelet introduced l’homme moyen, the average man. A decade later, applying the mathematics of astronomical error to the chest measurements of 5,738 Scottish soldiers, he found the familiar bell curve and concluded that the average was the type nature intended, and that each actual person was an error around it.

More than a century later, the United States Air Force tested that idea by accident. In the early 1950s, with pilots losing control of their aircraft at alarming rates, a young researcher named Gilbert Daniels measured more than 4,000 pilots on ten body dimensions and asked how many fell near the average on all ten. None did. A cockpit designed for the average pilot fit no pilot at all. The Air Force responded by requiring adjustable seats, pedals, and straps, and the cockpit began to fit the person.

The randomized trial’s central output is also an average. A hazard ratio of 0.40 is a statement about two groups. Within a treated arm, some patients benefit enormously, some not at all, and some are harmed, and the trial combines them into a single number that describes, in a precise sense, no one in particular.

The arithmetic is unforgiving. Daniels counted a pilot as average on a dimension if he fell within the middle 30% of the range. Were the ten dimensions independent, the chance of being average on all of them would be 0.3 multiplied by itself ten times: about six in a million. In high dimensions, nearly everyone is an outlier. A tumor described by hundreds of genomic, transcriptional, and microenvironmental features is a very high-dimensional object, and the patient who carries it is almost certainly an outlier on something that matters. A clinical guideline, distilled from the average results of trials, describes the center of a distribution in which almost no one actually lives. A physician trained only to follow it is trained, in effect, to treat a hypothetical patient who doesn’t exist.

II. A stratum of one

For most of the last half century, that limitation was tolerable because nothing better was available, but it’s becoming harder to accept because the gap between what we can do in the laboratory and what we can prove in the clinic widens every year.

Let’s consider the cost of proof. An analysis of public financial filings by Olivier Wouters and colleagues estimated that bringing a new medicine to market between 2009 and 2018 required a median of $985 million and a mean of $1.3 billion in research and development, including the cost of failures, with cancer drugs among the most expensive. Industry estimates run higher. In 2012, Jack Scannell and colleagues documented what they called Eroom’s Law, Moore’s Law spelled backward: the number of new drugs approved per billion inflation-adjusted dollars of research spending had halved roughly every nine years since 1950, an 80-fold decline. In oncology, an analysis of more than 21,000 compounds by my colleague Andrew Lo’s group at MIT found that only 3.4% of programs that entered phase 1 between 2000 and 2015 reached approval.

Now let’s think about the cost of knowledge. Sequencing a human genome cost roughly $95 million in 2001 and about $525 in 2022, and the price has continued to fall. A base editor for a single child’s mutation was designed, manufactured, and administered in six months. Individualized cancer vaccines are now made for each patient, one at a time, inside a phase 3 trial. The instruments of discovery have followed something like Moore’s Law. The instruments of proof have followed Eroom’s.

The standard evidence base itself rests on a small minority of patients. About 7% of adults with cancer in the United States enroll in treatment trials. The experience of everyone else, the overwhelming majority of people who receive cancer therapy, has been largely lost to systematic learning.

Precision oncology sharpened the mismatch rather than resolving it. Stratifying patients by biomarker made each average more relevant to the people inside it, but it also made each stratum smaller, slower to enroll, and more expensive per patient to study. The fixed cost of a conventional development program doesn’t shrink with the size of the population it serves. Followed to its logical end, the strategy arrives at a stratum of one, where the economics of the population model collapse entirely, and where many therapies that science could produce are never made at all.

III. The private life of a tumor

In 1976, the pathologist Peter Nowell proposed that a cancer is not a fixed entity but an evolving population. Every cell division is an opportunity for new mutations. Every therapy is a selective pressure. The tumor that survives treatment descends from whichever cells happened to carry a way out. Nowell’s model predicted that tumors would diversify over time, and that no two would diversify in quite the same way. He drew the clinical implication himself: each patient’s cancer, he wrote, may require individual specific therapy.

Five decades later, sequencing has confirmed both predictions. When Marco Gerlinger and colleagues sampled multiple regions of individual kidney cancers, they found that roughly two-thirds of the somatic mutations in each tumor were not present in every region of that same tumor. A single biopsy simply describes a neighborhood and not the city.

Targeted therapy made this evolution visible in real time. Imatinib transformed chronic myeloid leukemia, and resistance arrived through new mutations in the very kinase it inhibited. EGFR inhibitors in lung cancer selected for the T790M mutation; the drug designed to overcome T790M selected in turn for C797S, for MET amplification, and for transformation into entirely different histologies. BRAF inhibitors in melanoma were defeated by a long catalog of ways to reactivate the same signaling pathway. The pattern is general. The more precisely a drug strikes a single dependency, the more strongly it selects for cells that no longer need it, and the route of escape can differ from one patient to the next.

KRAS is the most recent and perhaps the most instructive example. For four decades it was the protein oncology could describe but not touch. Mutated in roughly one in five human cancers, it offered a nearly featureless surface to medicinal chemists, and its mutant forms held their activating molecule, GTP, with picomolar affinity. The word most often attached to it was undruggable.

In 2026 the protein finally yielded, and from two directions at once. The first was precision. In Nature Medicine, the oral KRAS G12D inhibitor GFH375 produced a 35.1% response rate in heavily pretreated pancreatic cancer, and HRS-4642, an intravenous liposomal G12D inhibitor, combined with chemotherapy produced responses in 63.3% of 30 untreated patients. The second was breadth. Daraxonrasib, which blocks the active form of RAS across mutant and wild-type variants, nearly doubled median overall survival in a randomized phase 3 trial in previously treated metastatic pancreatic cancer, from 6.7 to 13.2 months, and in August became the first RAS inhibitor approved by the FDA for the disease.

Nowell would have recognized what came next. The median duration of response to GFH375 was 5.6 months. When Mark Awad and colleagues at Dana-Farber examined patients whose cancers had escaped the KRAS G12C inhibitor adagrasib, they found secondary mutations in KRAS itself, bypass alterations in MET, NRAS, BRAF, and other genes, and in some lung cancers a change of identity from adenocarcinoma to squamous carcinoma. Several patients carried more than one escape mechanism at the same time. In October, a study of 242 patient-derived pancreatic cancer organoids identified a subset resistant from the outset to allele-specific and broad KRAS inhibitors alike, distinguished by silenced interferon signaling.

The lesson extends well beyond KRAS. A response rate tells us that a drug can open the gate. It cannot tell us which escape route a particular tumor will take, or when. That knowledge exists only at the level of the individual patient, and it changes over weeks.

IV. A medicine made for one

In February 2025, in a room at the Children’s Hospital of Philadelphia, an infant named KJ Muldoon received an intravenous infusion of a drug that had never been given to anyone else, and never will be. KJ was born with severe CPS1 deficiency, which left his liver unable to clear ammonia from his blood. A team led by Rebecca Ahrens-Nicklas and Kiran Musunuru developed a medicine designed around one of his two mutations: a guide RNA composed for that single sequence, a messenger RNA encoding a base editor, and a lipid shell to carry both into his liver. From diagnosis to first dose took about six months. The FDA cleared the application in about a week.

There was no control arm. There was no population. The number of patients in the study, then and forever, was one. KJ’s infusion was the most vivid point on a curve that has been bending for several years.

Months after KJ’s treatment, Vinay Prasad, former director of the FDA’s Center for Biologics Evaluation and Research, and former FDA Commissioner Marty Makary described what they called a plausible mechanism pathway. A bespoke therapy could reach the market without a randomized trial if it targeted a known molecular cause, if engagement of that target was confirmed, and if clinical improvement was observed across several consecutive patients, each serving where appropriate as their own control. They wrote with rare genetic disease in mind, but noted that the principles could extend to other drugs over time.

In oncology, the curve bent further this year. In the phase 3 INTerpath-001 trial, intismeran autogene, an mRNA therapy designed from the mutations in each patient’s own melanoma, reduced the risk of recurrence or death by 42% when added to pembrolizumab. Every patient in the experimental arm received a different molecule. What the trial validated was not just a product but a process: a way of reading a tumor and writing a medicine in response. Seven years earlier, the I-PREDICT study led by Jason Sicklick and Razelle Kurzrock had pointed the same way, finding that patients whose drug combinations more closely matched their tumors’ molecular profiles fared better.

The plausible mechanism pathway is a genuine advance, but it still carries a trace of Quetelet. It asks that a bespoke therapy act through a mechanism that generalizes, if only across a small group of patients who share a disease and its molecular cause. In cancer, that assumption gives way quickly. Two tumors with the same driver mutation diverge in their clonal architecture, their microenvironment, and their routes of escape, and the mechanism that decides the outcome in one patient may be irrelevant in the next. Here the N of 1 is not a degenerate case of a population. It is the unit of generalization. What carries from one patient to the next is not a molecule, and often not a single mechanism, but a method: the way a tumor is read, the way a therapy is designed in response, and the way its effect is measured and acted upon. A development system built on that insight would validate the method and let each patient supply the particulars.

Others have had to build that process themselves. The diagnosis Sid Sijbrandij received after his night in the emergency department set him on the standard path: surgery, radiation, and chemotherapy, with an estimated five-year survival of roughly 30%. I spoke with Sid, and with Jacob Stern, a geneticist and former 10x Genomics product leader who now leads the scientific enterprise behind Sid’s care, for a forthcoming episode of Precision Signals. What they described is the closest thing I have seen to individualized medicine practiced as a continuous loop of measurement and decision.

During that first course of treatment, a friend from his Y Combinator cohort, whose company was developing a cancer drug built on click chemistry, arranged for Sid to receive it through a single-patient investigational new drug application. The experience, Sid said, showed him that more was possible than the hospital and the trials. The lesson mattered in June 2024, when the cancer returned. His oncologist had no drug to recommend, and he qualified for no trial. He described himself as “default dead” and concluded that to survive he would “have to live in the future.”

Living in the future meant measuring everything: whole-genome and single-cell sequencing at each recurrence, organoids grown from his tumor, spatial transcriptomics, serial circulating tumor DNA, and experimental imaging agents wherever they existed. It meant combining therapies rather than queuing them; in the summer of 2024 he received five immunotherapies at once. It meant five single-patient applications to the FDA, personalized vaccines, and a radioligand aimed at fibroblast activation protein, chosen in part because his RNA sequencing data had flagged it. Jacob described the tumor as having built walls, an immunosuppressive microenvironment organized around cancer-associated fibroblasts. Longitudinal single-cell and spatial data from two biopsies and the final resection suggested that the radioligand brought those walls down and let tumor-reactive T cells through. The residual tumor was excised in April 2025, and the team has not found the cancer since.

One episode captures that loop better than any diagram. The team was developing a CAR-T therapy against B7-H3, a protein on Sid’s tumor cells. To learn where else the target lived in his body, he traveled to Beijing for an experimental B7-H3 imaging scan. The scan found no cancer, but his liver lit up about two and a half times more intensely than in the twenty patients scanned before him, a warning that the engineered cells might attack it. The therapy’s designer responded with a logic gate: the cells would act only when they also encountered fibroblast activation protein, abundant in Sid’s tumor and nearly absent from the liver. A single measurement, in a single patient, redesigned a therapy before it was ever given.

The economics have moved as well. When first exploring making a personalized mRNA vaccine, the total cost of piecing together manufacturing and components would have mounted into the millions; a company Sid and Jacob later helped start can now make such vaccines at roughly a tenth of that cost. And true to the ethos that built GitLab, Sid has published the complete record, roughly 37 terabytes of genomic, imaging, and pathology data, at osteosarc.com.

Jacob was careful about what this proves. Causation cannot be established in an N of 1, and Sid’s resources make his path difficult to reproduce. But the case demonstrates something the population model was not built to see: what care looks like when the loop between observation and action is short, dense with data, and free to follow the tumor wherever it goes.

V. The loop

We have tended to locate precision in the molecule. A KRAS G12D inhibitor is called precise because it binds one allele; a pan-RAS inhibitor is called broad because it binds many. But the cases above suggest that precision is better understood as a property of the system in which a molecule is used, and that this property has at least three components:

• Specificity: how narrowly an intervention matches the biology of the tumor.

• Resolution: how much of the tumor’s state we can see, in depth (single-cell, spatial, circulating DNA) and in breadth (across lesions and through time).

• Tempo: how quickly we can act on what we see, relative to how quickly the tumor changes.

The precision of care behaves less like the sum of these terms than like their product, and the weakest one limits the whole. A highly selective drug given on a fixed schedule, with scans every eight weeks and a change of course only after a tumor has visibly grown, combines exquisite specificity with poor resolution and slow tempo. It selects for resistant clones that remain invisible until they dominate. A broad drug adjusted against frequent molecular and imaging readouts can, paradoxically, deliver more individualized care than a narrow one used statically.

Each term exacts its own price. Specificity buys tolerability and the freedom to combine, at the cost of stronger selective pressure and smaller markets. Resolution buys early warning and new targets, at the cost of money, noise, and the labor of interpretation. Tempo buys adaptation before resistance consolidates, at the cost of causal clarity and regulatory friction.

Seen this way, the long argument between precision and breadth dissolves into something more tractable. The case for breadth is a hedge against low resolution and slow tempo: if escape cannot be seen coming, suppress every route at once. The case for the N of 1 is a bet that resolution and tempo can be raised so far that specificity becomes an asset rather than a liability. Both positions are rational. They disagree about the loop, and the loop is the one thing the current development system neither measures nor regulates.

VI. The witnesses

In 1962, the only reliable witness to a drug’s effect was the control arm. Clinical records were handwritten, outcomes were recorded inconsistently, and a patient treated outside a trial disappeared from the evidentiary record the moment she left the clinic. Requiring randomization before approval was, among other things, a recognition that the clinic could not testify.

That is no longer true, and the change has happened quietly.

The tumor itself can now be interrogated repeatedly. Circulating tumor DNA reports on tumor burden and emerging resistance alleles from a blood draw, at intervals of weeks rather than months. Single-cell and spatial methods, which a few years ago belonged to research laboratories, have entered the care of individual patients. Quantitative imaging can measure the growth kinetics of each lesion, rather than reducing a scan to a single category of response.

The record has also become more interpretable. Large language models can now read unstructured clinical notes and extract endpoints that once required teams of trained abstractors. Researchers at Flatiron Health reported that model-extracted progression events across 14 cancer types approached the accuracy of expert human abstraction and yielded nearly identical estimates of real-world progression-free survival. Ambient documentation, which drafts the clinical note from the conversation itself, is capturing reasoning and symptoms that previously went unwritten.

And the past has become available as a comparator. Repositories of historical clinical trial data, including control arms, allow an individual trajectory to be placed against what similar patients experienced on standard therapy.

Patients have begun to keep the record themselves. When Jacob Stern first met Sid, Sid sent him a shared folder of some 2,500 documents and a running file of notes from conversations with clinicians and scientists that had already reached 700 pages. That archive is an example of what every patient’s course could leave behind if it were captured by design rather than by personal determination.

This points to a different idea of how medicine learns. For nearly two decades, the learning health system, a phrase popularized by an Institute of Medicine roundtable in 2007, has been imagined as an infrastructure problem: connect every clinic through electronic health records, aggregate the data, and let insight rise from the database. The records were built, at a cost of billions of dollars, but they were designed for billing and documentation rather than discovery, and a 2016 time-and-motion study found that physicians spent nearly two hours on the record and desk work for every hour of direct clinical time with patients. The learning that was promised has been disappointing at best. The N of 1 suggests a different anchor. The unit that joins observation, decision, and outcome is not the database. It is the treating clinician, who orders the measurement, interprets it, chooses the therapy, and sees what happens next. Bring research to that point of care, equip the clinician to measure and record as an investigator would, and the learning health system stops being an aspiration about data and becomes a description of practice.

The first generation of real-world evidence promised much and delivered less, largely because the data were sparse and the extraction manual. The second generation will be different in kind. A patient treated outside a trial no longer needs to be a lost observation. With measurements specified in advance and modern tools to read the record, every individualized course of treatment can become a precise data point.

VII. A regulator for the N of 1

Before a drug is approved, the FDA presides mainly over safety: to open a standard investigational trial, a sponsor need only show that the study will not expose patients to unreasonable risk, not that the drug works. The same logic governs expanded access, the pathway a physician uses to request an unapproved drug for a single patient outside a trial. There, the agency asks whether the potential benefit justifies the risk and whether better alternatives exist, and from fiscal years 2012 through 2015 it allowed 99% of nearly 5,800 expanded access requests to proceed, usually answering emergency requests within hours. The regulator is rarely the bottleneck.

What fails is everything around it. Manufacturers face the friction of supplying unapproved drugs, institutions add their own layers of review and contracting, and no one is responsible for recording the outcome. Expanded access was designed for compassion, not investigation; each course is treated as a one-off exception, and almost nothing is aggregated. The Right to Try Act of 2018 bypassed the FDA in the name of access, but it did nothing to help us learn from individual patient experiences.

The deeper problem, I believe, is categorical. Our regulatory system recognizes two objects. One is the product: a fixed molecule, manufactured identically and approved for a population. The other is the trial: a bounded experiment with a protocol, an enrollment target, and an end date. N of 1 care is neither. Its object is a practice: a clinician, a patient, and a loop of measurement and decision that changes from week to week and does not end on a schedule. Asking the FDA to govern that practice as a product or as a trial is asking it to regulate the wrong thing.

Medicine has faced this problem before, and solved it three times.

• Surgery has always been N of 1. Every operation adapts to a particular body, and no new technique requires FDA approval. Oversight lives instead in credentialing, peer review, and outcome registries. The Society of Thoracic Surgeons National Database, begun in 1989, captures more than 90% of adult cardiac surgery programs in the United States and benchmarks each against risk-adjusted expectations.

• Blood and marrow transplantation is individualized by donor, conditioning regimen, and immune recovery. Federal law, most recently the TRANSPLANT Act of 2021, requires U.S. centers to report every allogeneic transplant to a national outcomes database, and each center’s risk-adjusted one-year survival is published every year.

• Laboratory testing is regulated through the laboratory rather than the test. Under CLIA, a certified laboratory may develop and run its own assays; nearly 80,000 such tests are offered by almost 1,200 laboratories, and in 2025 a federal court confirmed that this practice-based framework, not FDA product review, governs them.

In each case, society accepted innovation at the level of the individual case because oversight moved from the object to the practitioner, the process, and the record. Applied to individualized cancer therapy, the same logic yields an architecture with four parts.

License the practice. Centers that treat patients on N of 1 regimens would be accredited, much as transplant centers are, and the clinician-investigators within them credentialed. Accreditation would require a molecular tumor board, pharmacologic and safety expertise, defined monitoring, and the infrastructure to capture data as care is delivered. Accredited centers would hold standing protocols, so that the FDA’s role shifts from approving each regimen to setting the boundaries within which regimens may proceed.

License the process. For an investigational agent, the sponsor and the FDA would agree after phase 1 on a safety envelope: the doses, combinations, and monitoring rules within which accredited centers may use the drug on notice rather than application, with the agency free to intervene at any point. For bespoke products, the object of review would be the validated system that designs and manufactures them. Platform technology designation, created in 2022, points in this direction but applies only to platforms already incorporated in an approved product. An individualized therapy license would let each patient-specific product made under a validated process be released after defined quality checks rather than through a new application.

Mandate the record. As in transplantation, every course at an accredited center would be reported to a national registry under a minimum data standard: the molecular rationale and comparator recorded before the first dose, imaging and circulating tumor DNA at fixed intervals, outcomes extracted from the record with validated methods, and failures reported as faithfully as successes. Within-patient comparisons, such as the progression-free interval ratio Daniel Von Hoff and colleagues used in 2010, help detect signals but cannot stand alone. Their strength comes from accumulation across many patients, analyzed with hierarchical methods specified in advance and set against historical controls. Center-level outcomes would be published, as they are for transplant centers.

Let the label live. The FDA would read the registry continuously. When evidence for an agent, or for a validated process, in a defined molecular context crosses a threshold set in advance, the agency would grant a provisional indication that expires by default unless continued accrual or a randomized study confirms it. The nearest existing relative is accelerated approval, which the Food and Drug Omnibus Reform Act of 2022 strengthened with authority to require confirmatory trials at approval and expedited procedures to withdraw. The difference is that evidence would arrive continuously from practice rather than episodically from trials, and the label would state not only the indication but the strength of the evidence behind it.

Each participant’s role changes accordingly:

Parts of this can begin now. Accredited centers can hold standing protocols under existing expanded access regulations, with a single review board and standard contracts. ASCO’s TAPUR study has run a working prototype since 2016, matching patients with advanced cancer to targeted drugs used outside their labels, with drugs contributed by manufacturers, review by a molecular tumor board, and results reported publicly, negative findings included. The FDA’s 2017 guidance on adverse events from expanded access, which noted that the agency knew of no case in which such events had blocked an approval, can be made explicit and binding for N of 1 protocols.

Full implementation and institutionalization would likely require Congress to create a statutory framework that borrows from transplant oversight while preserving the FDA’s authority over drug safety and manufacturing quality. Such a law would authorize accreditation and mandatory reporting with explicit privacy safeguards; create an individualized therapy license for validated design and manufacturing processes; and establish provisional indications that expire unless their evidence requirements are met. It would also give manufacturers narrowly defined liability protections and incentives, such as extended exclusivity or a priority review voucher, conditional on standing access and data sharing; fund the diagnostics, imaging, and clinician time required by the minimum data standard; and give patients a statutory right to direct the preservation and transfer of their own tissue and data.

Imagine how this would look from the clinic:

A woman with KRAS G12D pancreatic cancer progresses after first-line chemotherapy at a community hospital affiliated with an accredited center. Her sequencing and baseline scan are already in the record, and the center’s molecular tumor board reviews them that week. An investigational G12D inhibitor, past phase 1, is available under a standing authorization from its manufacturer. Her oncologist pairs it with an EGFR antibody, within the drug’s safety envelope, because her tumor’s profile suggests feedback through that receptor. The FDA receives a notification, not an application. Her scans and blood draws follow the registry’s schedule, and her outcomes are recorded without anyone filling out a form. She is one of four hundred such patients across sixty sites. When the pooled signal crosses its threshold, the agent receives a provisional indication in a precisely defined molecular context while a confirmatory trial enrolls. Her treatment was individualized. Her experience counted.

Several of these ideas echo what Sid argued from the other side of the system: that investigators should be free to choose their review board, that a drug which has completed phase 1 should reach patients whose DNA and RNA make them strong candidates, and that controls built for millions of doses should not be imposed on ten. Engineers can argue about architecture in the abstract indefinitely, he said, but show them two pieces of code and most will agree on which is better. Regulations change the same way: when a specific patient is standing at the door and the rule plainly does not fit.

None of this lowers the bar for safety. It changes what the regulator regulates. Since 1962, the object of oversight has been the molecule. In an N of 1 system it becomes the loop: the practitioner who runs it, the process that supplies it, the record it leaves behind, and the patient waiting on the other side.

VIII. The ledger of doubts

A proposal to reorder questions that were written into law after two tragedies deserves the skepticism those tragedies earned.

Conditional approvals can outlive their evidence. In 2014, Japan created a conditional, time-limited approval pathway for regenerative medicine, allowing products to reach patients on evidence of safety and probable benefit. Its first product, an autologous myoblast sheet for heart failure, was approved in 2015; nearly a decade later, regulators concluded that it had not met the criteria for full approval. Conditional authorization works only when its deadlines are enforced, its success criteria are fixed in advance, and failure carries consequences.

Concurrency obscures causation. When several interventions are given together and changed quickly, assigning credit to any one of them is difficult. Within-patient controls and hierarchical pooling narrow the uncertainty but do not eliminate it. The N of 1 pathway should feed randomized trials, not replace them, whenever pooled experience produces a signal strong enough to test.

The survivors write the history. Successes are visible; failures rarely are. A registry that records every course, including those that end badly, is the only defense against a literature composed of the lucky.

Desperation can be exploited. Patients with advanced cancer are vulnerable to unproven therapies sold at high prices. Qualified institutions, independent review, and public registration of every protocol are what separate a development pathway from a marketplace.

Access can become privilege. The most visible N of 1 successes so far have depended on extraordinary means. Jacob Stern named the ingredients of Sid’s path plainly: resources, risk tolerance, agency, and time. A pathway open only to a few would widen the very inequities that population medicine, for all its limitations, was designed to narrow. Some barriers persist at any price. Sid found it difficult even to obtain his own tumor tissue, which hospitals typically fix in paraffin, limiting later analyses, and are reluctant to release; Even One Ventures, the firm he and Jacob founded, started a company for that problem alone. The goal must be, and can be, a loop that grows cheaper, and simpler, with every patient who passes through it.

IX. What the average cannot know

Quetelet believed that the average man was the truth and that each living person was an error around it, an imperfect anomaly that nature did not intend to create. Medicine inherited a version of that belief. We built our evidence on the assumption that the signal lives in the population and the individual is noise to be averaged away.

What I’ve described here suggest the opposite. Every tumor carries a private history of mutation, adaptation, and escape that no average can contain. In modern medicine, the individual is the actual signal and the average is a mere approximation.

This doen’t make the randomized trial obsolete, any more than adjustable cockpits made aeronautical engineering obsolete. It means that the trial is one instrument among several, and that for the patient who has run out of options, it may be the wrong instrument to wait for. The alternative is not to abandon evidence. It’s to stop discarding it: to treat each individualized course as an experiment, record it as carefully as we would record a trial, and let the pattern emerge from the accumulation of single cases.

If the individual patient is the unit of generalization, the arithmetic of waste changes as well. Every patient treated without measurement is an experiment run and discarded. Every outcome left unrecorded is knowledge withheld from the next person with a similar tumor. The population model taught us to accept the control arm as the price of rigor. The N of 1 asks us to see the unmeasured patient as the price of bad habits.

KJ Muldoon will grow up as the only person who has ever received his medicine. Whether he is also the last person to benefit from it depends on whether we learn to read what his case, and thousands like it, can teach.

Sid Sijbrandij has now gone more than a year without detectable cancer. His team still images him regularly and tracks his disease status through blood-based minimum residual disease monitoring tests. Personalized vaccines continue to educate his immune system, and backup therapies stand ready in case the disease returns. His motto is two words: stay paranoid.

There is no finish line in the N of 1, only a loop that keeps turning: measure, decide, act, and measure again:

Medicine built for the average could offer an ending: a hazard ratio, a label, a line in a guideline. Medicine built for the individual offers something less final and more demanding, which is attention, sustained for as long as the disease keeps changing. A medicine for no one in particular was one of the great achievements of the last century. A medicine for someone in particular is harder, and it has no last page.

We’ve spent decades training physicians to follow guidelines. Today, we can, and must, follow the patient: to reason about one person’s data with the rigor once reserved for populations, to treat the outlier as the rule rather than the exception, and to go where the evidence leads.

For sixty years we have asked whether a drug works. The question for the next sixty should be whether we’re willing to learn from one patient at a time, for as long as it takes, and to let what we learn count as evidence.

Subscribe now

I. Speculative science

Lloyd Old of Memorial Sloan Kettering (MSK), one of the founding figures of cancer immunology, once submitted a grant. The reviewer, an eminent scientist, dismissed more than the proposal and labeled the entire field speculative science.

Jedd Wolchok, now Meyer Director of the Sandra and Edward Meyer Cancer Center at Weill Cornell Medicine, told me this story when he joined me on Precision Signals. He was a fellow at MSK in the late 1990s, a period he describes as the epitome of nihilism around cancer immunology. People spoke about the field the way they spoke about mysticism. “You either believed in it or you didn’t,” he said.

What struck me is that Jedd never described his career as an act of belief. He described it as an inability to let go of what he had seen.

Early in his training, he worked in the laboratory of Alan Houghton, a physician-scientist in Old’s group who was testing antibodies as cancer therapeutics. There he watched patients with metastatic melanoma receive R24, a mouse antibody against the GD3 ganglioside, a molecule abundant on the surface of melanoma cells. It was one of the first clinical trials of a monoclonal antibody in a solid tumor. Patients soon developed antibodies against the mouse protein, and the window of activity closed quickly. But in the first published report, three of 12 patients had major tumor regressions. An immune product, not a cytotoxic drug, had made cancer shrink.

Years later, as a fellow, he immunized mice against tyrosinase and related proteins shared by melanoma and the normal pigment cells it arises from. The vaccinated mice resisted an intravenous melanoma challenge. The immune system could be taught to recognize a cancer as foreign, even when its targets were the body’s own proteins. He resists calling these moments anecdotes. They were, in his words, singular observations, and for him they were enough.

II. Bigger before smaller

Those observations carried him into the early clinical studies of CTLA-4 blockade. CTLA-4 is an inhibitory receptor on T cells, one of the molecular brakes that keep immune responses in check. Ipilimumab, an antibody that blocks it, was designed to release that brake. The patients who enrolled had advanced melanoma, a median survival of about six and a half months, and often one, two, or three failed therapies behind them. Ipilimumab produced striking responses in 10% to 15% of them, and those responses tended to last.

The pattern of response, however, looked nothing like chemotherapy. With chemotherapy, a response usually appears early or not at all. With ipilimumab, some patients’ tumors grew dramatically for weeks, sometimes a couple of months, before shrinking rapidly. Under the WHO and RECIST criteria, the “gold standard” for tumor response assessments, a growing tumor meant the drug had failed and the patient should come off study.

Jedd and his fellow investigators began to notice a discordance. Some patients whose scans looked terrible were feeling better. They brought these observations to Bristol Myers Squibb, the company developing ipilimumab, and colleagues there agreed to let them keep such patients on study as long as no better therapy was available. “And of course, there were no better therapies for those people to receive,” Jedd said. The investigators continued to follow these patients, and about a quarter of those with overt radiographic progression eventually responded or stabilized with no additional therapy.

The scans were not wrong. The rules for reading them were. In 2009, Jedd and his colleagues published the immune-related response criteria, which measured total tumor burden and allowed for growth before shrinkage. Under the new criteria, a single new lesion no longer erased the disappearance of many others.

When ipilimumab was approved in 2011, the evidence that carried it was overall survival, because response rates and progression-free survival had proven unreliable measures of what the drug was doing. PD-1 blockade followed. Because CTLA-4 and PD-1 restrain T cells through non-redundant mechanisms, Jedd went on to lead the clinical development of their combination, which made immunotherapy a standard of care in melanoma.

III. The problem was always there

I came to the same problem from the regulatory side. In 2016, at FDA, we compared two readings of the same scans across registrational trials: the investigators’ assessments and those of blinded independent radiologists. Across 13,677 paired assessments, they disagreed about 30% of the time. In tumors that are hard to measure, such as pancreatic and ovarian cancers, the disagreement exceeded 40%.

Some of that disagreement is built into the criteria. RECIST reduces a tumor to the summed diameters of a few selected lesions. The sum must shrink 30% to count as a response, and it must grow 20% from its smallest value to count as progression. Everything in between is called stable disease, whether or not anything about it is stable.

Immunotherapy did not create this measurement problem. It exposed it. As Jedd put it, an imaging study holds far more information than a binary of bigger or smaller, including the shape and density of each lesion, the features radiomics tries to capture. Cell-free DNA shed by tumors into the blood may offer an earlier read while the images lag. Both point toward measuring the disease as it changes over time, rather than as a single size at a single moment.

IV. How much

Fifteen years after the first checkpoint inhibitor approval, there is still no gold-standard biomarker to predict who will benefit. Tumor mutational burden comes closest in principle. The more mutations a tumor carries, the more neoantigens it may display, the mutated protein fragments that make it look foreign to T cells. Jedd calls it a good but not great biomarker. What the field needs, he argues, is a precise measure of the fitness of individual neoantigens: how well each is presented on the tumor surface, how widely it is shared across tumor cells, and how likely T cells are to recognize it. He sees this as a natural use case for AI.

Better prediction leads to the question he keeps returning to. It is no longer whether immunotherapy works, but “when and how much immunotherapy an individual patient needs.” The neoadjuvant trials have already supplied part of the answer. The older sequence was surgery first, then immunotherapy to eliminate residual disease. It has been outperformed by treating while gross disease is still present, resecting what remains, and letting the pathologic response guide whether more therapy is needed.

Near the end of our conversation, I offered a version of the principle in surgical terms: the more we know, the less we cut.

“That’s right,” he said.

V. On us

Answering these questions depends on who participates in the trials that ask them. At Weill Cornell, Jedd is opening trials at sites in Brooklyn and Queens and choosing studies whose eligibility criteria reflect the communities those sites serve. “People are interested in clinical research,” he told me. “We just have to make it much more convenient for them.” Then he added, “That’s really on us.”

Cancer immunology has rarely looked like progress at first. It looked like mysticism, and then it looked like a failed scan. The work now is to measure well enough that we recognize progress when it arrives, and to make sure more patients are in the trials where it does.

Subscribe now

I. Three experiments

Take a single chest CT from a patient with metastatic non-small cell lung cancer. Give it to an AI-enabled measurement device cleared five years ago, one that segments the target lesions and returns the sum of their longest diameters. Run it a thousand times. You will get the same number a thousand times. That is what the device was built to do, and it is what FDA was built to evaluate.

Now give the same scan to a generative AI system that reads the images, drafts the radiology report, and characterizes the response. Run it a thousand times. You will get a distribution of reports. Most will agree. Some will phrase the findings differently. A few may disagree on whether the disease has progressed.

Then give the scan to a thousand radiologists.

I know what happens in the third experiment. Radiologists reading the same images disagree in roughly 30% of cases. Not on marginal details. On the response category that determines whether a patient stays on a drug, and whether a drug stays in development.

The question these experiments raise is not which of the three readers is reliable. It is which of the three we have been treating as the standard, and why.

In August, FDA’s Center for Devices and Radiological Health (CDRH) released a discussion paper titled Considerations for the Regulation of Generative AI-Enabled Medical Devices. On its surface, it is a request for feedback on how to evaluate a new class of software. The agency is careful to say that it is not guidance and does not propose policy. Read closely, it is more consequential than that. It is the first document in which the agency concedes that some medical devices will have to be evaluated less like instruments and more like clinicians.

That concession carries a consequence the paper does not state, and one the agency may not have anticipated:

If we are going to judge machines the way we judge clinicians, we will soon have to judge clinicians the way we judge machines.

II. What the paper proposes

The paper starts from a premise the agency first laid out at its Digital Health Advisory Committee meeting in November 2024. Generative AI devices accept open-ended inputs, perform multiple subtasks, and produce variable outputs to similar inputs. Many are built on foundation models developed by third parties who disclose little about their training data or evaluation. Some change after deployment through updates to the model, the prompts, the retrieval strategy, the guardrails, or the orchestration logic. The traditional approach of testing a device across a representative sample of inputs and outputs does not scale to a system whose input space is effectively unbounded.

From there, the paper sketches four ideas.

The first is a two-axis risk framework. One axis describes what the device does, moving from non-directive information to action-directing information, then to actions taken under clinician supervision, and finally to fully autonomous action. The other axis describes the consequence of relying on an incorrect output. A risk score for a future cardiovascular event sits in a different place than an instruction to change a basal insulin dose. Within this framework, the paper makes several notable observations. Directiveness is a continuum, not a binary. A “talk to your doctor” disclaimer does not make an action-directing output less directive. Measurement functions may carry higher risk than their non-directive character suggests, because the user usually cannot check the basis of the output. Patient-facing outputs may warrant different treatment than clinician-facing ones. And conversational devices can drift from informing to directing over the course of a single exchange.

The second is a competency-based approach to premarket evaluation, explicitly inspired by how clinicians are trained and credentialed. It has two parts. Device benchmarking would test the deployed system, not the foundation model alone, across safety, clinical proficiency, generalizability, and, for agentic systems, the ability to plan and act within a defined envelope. Clinical confirmation would then establish performance in real or clinically representative use, through methods ranging from retrospective evaluation and shadow deployment to standardized patient interactions, clinician adjudication, and prospective trials. Performance might be compared against a panel of clinicians, a median clinician in practice, or, where one exists, a conventional reference standard.

The third is greater reliance on the postmarket period. The agency asks whether it should accept more uncertainty at authorization in exchange for periodic re-benchmarking, sampled clinician review, and drift monitoring after deployment, with the premarket benchmark serving as the baseline against which future changes are judged.

The fourth is a voluntary Foundation Model Master File, through which model developers could share architecture, training data provenance, known failure modes, and update commitments with FDA on a confidential basis, for reference by device sponsors.

The paper closes with 26 questions. They are largely good questions. Several are harder than the paper lets on.

III. The wrong word

Much of the public conversation about AI in medicine turns on a single word: deterministic. Vendors market “deterministic AI” to hospitals and payers as a safer category of product. Critics argue that large language models are unsuitable for clinical use because they are not deterministic. Both sides use the word to mean several things at once: identical outputs for identical inputs, reproducible results that can be audited, predictable behavior across situations, or logic written by people rather than learned from data. These properties are related but distinct, and a system can have one without the others.

The FDA paper does not use the word, to its credit. Instead, under a benchmarking element on robustness, reliability, and reproducibility, it breaks variability into components that can actually be tested: consistency across repeated runs, across paraphrases that reflect differences in health literacy or emotional framing, across the order in which information is presented, across missing or contradictory inputs, and across long conversations. It then draws a line that matters more than it appears to. Variation in phrasing may be acceptable. Variation in safety-critical behavior, such as a diagnostic conclusion, is a failure.

Without naming it, the agency has imported the language of psychometrics. What the paper describes is test-retest reliability, or more precisely, intra-rater reliability for a machine. Medicine already has a mature toolkit for this: kappa statistics for categorical agreement, intraclass correlation coefficients for continuous measures, and limits of agreement for measurement methods. We use these tools to evaluate radiologists, pathologists, and the endpoints that drug approvals depend on.

This suggests a simple proposal. Generative AI devices should report their reliability on the same scales we use for human readers, and their safety-critical outputs should be held to prespecified agreement thresholds. That would replace an argument about vocabulary with a measurement.

It would also make visible a principle every psychometrician learns early: reliability is necessary for validity, but not sufficient. A system can return the same wrong answer every time. In some ways that is the most dangerous failure, because consistency reads as confidence. The paper gestures at this when it notes that measurement functions may carry elevated risk because the user cannot evaluate the basis of the output. A tumor measurement that is perfectly reproducible and systematically biased will pass every repeatability test and still mislead every clinician who relies on it.

There is one place where the vocabulary problem could still cause harm. If what makes a device “generative” for regulatory purposes is its variability, a developer may argue that an LLM-based device configured to sample greedily, with structured outputs, is effectively deterministic and should be evaluated like conventional software. That argument should not succeed. Fixing sampling settings removes one source of variation, and not reliably: research from Thinking Machines Lab in 2025 showed that even nominally deterministic inference can vary with how a server batches requests. It does nothing about paraphrase sensitivity, distribution shift, or a foundation model that changes upstream. The question is never whether a device is deterministic. It is whether its clinically consequential behavior is reliable, valid, and stable, and how we know.

IV. The variable reference

The most consequential passage in the paper is a short section on standards of performance. Conventional device evaluation compares an output to a known correct answer. For many generative devices, no single correct answer exists, and many responses may be acceptable. The paper therefore proposes comparing the device to a panel of qualified clinicians whose consensus reflects the standard of care, or to a median clinician in practice.

This is a reasonable choice. It is also the point at which the evaluation of machines becomes an evaluation of medicine.

To set an acceptance criterion against a clinician comparator, you have to know how clinicians perform. Not in aggregate, and not by reputation, but on the same cases, under the same conditions, measured with the same rigor applied to the device. We rarely have that information, and not for lack of spending. Between 2008 and 2018, CMS invested billions in developing quality measures, and only about a third of the 2,266 it produced were ever put to use. CMMI, which receives $10 billion each decade to test new ways of paying for care, spent $7.9 billion operating its models in its first ten years, several of them built around bonus payments for meeting quality and other metrics. They all mostly failed. After billions of dollars and more than a decade of measurement, we still have no systematic way to know whether a clinician made the right call on a given case, or how often two clinicians looking at the same patient would make the same one. That is exactly what a clinician comparator requires, because the panel used to judge a device will be drawn from clinicians who often disagree with one another.

The machine is the first clinical reader that can be run a thousand times on the same case, and that reverses the asymmetry. We will soon have reliability estimates for devices far more precise than anything we have for the clinicians they are compared to. At that point, the comparator becomes the weakest part of the evaluation. Consider a hypothetical device that agrees with a three-person panel 85% of the time. It sounds deficient until we learn that the panelists agree with each other only 78% of the time.

The choice of comparator is therefore not a technical detail. It encodes a value judgment. Benchmarking against the median clinician accepts that half of practicing clinicians perform below the standard. Benchmarking against expert consensus may set a bar most of medicine does not meet. Benchmarking against specialists penalizes devices built to extend specialist knowledge to places where no specialist is available.

This is why the most important question in the paper may be Question 15, which asks whether performance could be assessed relative to the care likely to occur in the absence of the device: unaided clinical judgment, delayed specialist review, or no intervention at all. For a patient in a county without an oncologist, the alternative to a device is not a tumor board. It is a long drive, a long wait, or nothing.

Taken seriously, this has an uncomfortable implication:

The benefit-risk profile of a generative device is not a property of the device alone but of the device in a particular care environment.

The same system could be net beneficial in a rural primary care clinic and net harmful in an academic cancer center, where it displaces better judgment. If so, deployment context belongs in labeling, and benchmark datasets must match the settings where the device will be used. The paper hints at this when it notes that benchmark data should represent deployment contexts. It is worth saying directly: intended use may need to include intended setting.

V. The conversation is the device

Since the Medical Device Amendments of 1976, device regulation has been anchored to intended use. The manufacturer declares what the device is for through its labeling and claims, and FDA evaluates it against that declaration. The concept assumes that the manufacturer’s intent fixes the device’s function. A glucose meter measures glucose. It does not decide, halfway through a reading, to recommend a change in insulin.

A conversational device can. The paper acknowledges this directly. A device may begin by providing general information and migrate, over the course of an exchange, into directing action. It asks how intended use could be characterized when a device’s behavior is emergent across a conversation.

This is a deeper problem than it appears. In a conversation, the user co-authors the function. The same device, with the same labeling, will perform different functions for a patient who asks a general question and for one who keeps pressing until the device tells them what to do. Intended use, in the traditional sense, becomes something closer to intended scope: a boundary the manufacturer tries to hold, not a function the manufacturer defines.

The FDA paper’s treatment of disclaimers points in the same direction. It suggests that whether an output is directive depends on its substance and context, not on whether it uses words like “recommend” or “consider,” and that a closing line telling patients to consult their doctor does not make an instruction less of an instruction. This is a quiet but significant shift. For decades, a great deal of health software has relied on labeling and disclaimers to stay outside device oversight. The paper suggests that the regulated object is no longer the label. It is the utterance, and the sequence of utterances.

The philosopher J.L. Austin argued that some statements do not describe the world but act on it. “I recommend increasing the dose” is not a report. It is an act, with consequences. Generative devices produce such acts at scale, in response to users the manufacturer will never meet.

If that is the regulated object, then intended use must be defined statistically. I would propose thinking of it as an envelope: the set of conversational trajectories the device is authorized to support, together with a measured rate at which realistic trajectories stay inside it. Benchmarking would then include adversarial and naturalistic multi-turn testing designed to estimate that containment rate, with acceptance thresholds tied to the consequence axis. A device that stays in scope 99.9% of the time may be acceptable for general health information and unacceptable for anything that touches dosing. The paper already points toward this with its benchmarking element on scope maintenance, which tests whether cumulative interactions drift out of scope even when each turn appears in scope. The next step is to make the envelope, and its measured integrity, part of the device’s identity.

VI. Which device

The clinician analogy at the center of the paper has a limit that deserves more attention. A physician who passes their boards on Monday is the same physician on Tuesday. She carries her training, her memory, her license, and her liability with her. Her identity is continuous, and our entire system of credentialing depends on that continuity.

A generative device has no such continuity. It is a stack: a foundation model, a fine-tuned layer, a system prompt, a retrieval corpus, a set of guardrails, orchestration logic, and a user interface. Each layer can change. Some changes are deliberate and built in. Some happen continuously by design. Some are made by a foundation model developer the sponsor does not control, on a schedule the sponsor may learn about after the fact. The paper names all three. What it does not quite ask is the underlying question: when every layer can change, what exactly was authorized?

The paper’s answer, implicitly, is behavioral. The premarket benchmark becomes a baseline, and future versions are re-benchmarked against it. I think this is the right instinct, and it can be made more precise by borrowing from a domain FDA already knows well.

Generic drugs are not required to be molecularly identical to the reference product in every respect. They are required to be bioequivalent. The standard is explicit and quantitative: the 90% confidence interval for the ratio of key pharmacokinetic measures, such as peak concentration and total exposure, must fall within 80% to 125% of the reference. Two products that meet that standard are treated as interchangeable for regulatory purposes, even if their manufacturing processes differ.

The same logic could define identity for generative devices. A device would remain the authorized device as long as its behavioral profile, measured on a sequestered benchmark, stays within prespecified behavioral equivalence margins on the dimensions that matter: safety-critical escalation, refusal, diagnostic agreement, measurement bias, and subgroup performance. A change of any kind, at any layer, by any party, would trigger an equivalence test. Within the margins, the change is documented. Outside them, it is a new device, or at least a change that requires review.

This framing has three advantages. It is indifferent to where a change originates, which matters when the change comes from a third-party model update. It gives Predetermined Change Control Plans a quantitative core that does not require anticipating every future modification, which the paper identifies as a central difficulty. And it moves the debate about what counts as a significant change away from architecture, which is hard to reason about, toward behavior, which is what patients actually experience.

It also forces a design choice that is easy to defer. Equivalence margins must be set before anyone knows how the next version will behave, and they must be set by clinical consequence. A 2% shift in phrasing style is irrelevant. A 2% shift in the rate of missed escalations for chest pain is not.

VII. The postmarket bargain

The paper asks whether FDA should accept greater uncertainty at authorization in exchange for stronger monitoring after deployment. It frames the question as new. Drug development has been running this experiment for more than three decades.

Accelerated approval, established in 1992, allows drugs for serious conditions to reach patients on the basis of a surrogate endpoint reasonably likely to predict clinical benefit, on the condition that the sponsor confirm that benefit afterward. It has delivered important therapies years earlier than traditional approval would have. It has also produced a well-documented pattern: confirmatory obligations slipped. A 2022 review by the HHS Office of Inspector General found that more than a third of accelerated approvals had incomplete confirmatory trials, many past their planned completion dates. Indications that failed confirmation sometimes remained on the label for years. In 2021, the agency and its Oncologic Drugs Advisory Committee had to work through a series of these so-called dangling indications publicly, several of them in checkpoint inhibitors. It took legislation, the Food and Drug Omnibus Reform Act of 2022, to give FDA the authority to require confirmatory trials to be underway before approval and to streamline withdrawal.

The lesson was not that the bargain was a mistake. It was that a bargain is only as good as its enforcement, and that enforcement must be designed in at the moment of authorization, when the agency has leverage, not negotiated afterward, when it does not.

CDRH has an opportunity to learn that lesson without repeating it. If generative devices are to be authorized with greater premarket uncertainty, several conditions seem essential. The monitoring program should be operational on the day of deployment, not planned for a later date. The triggers for re-benchmarking, review, or restriction should be prespecified, quantitative, and linked to consequences that follow automatically when thresholds are crossed. Results should be reported publicly at a defined cadence, so that clinicians and health systems can see how a device performs after authorization. And the obligation should attach to the sponsor regardless of where a change originates.

Software also offers something drugs never did. A drug’s postmarket experience reaches the agency slowly and incompletely, through adverse event reports and studies that take years. A deployed generative device can, in principle, be observed continuously. Every interaction can be logged, sampled, and adjudicated. The paper raises the possibility of machine-based supervisory agents to do some of this work. That is plausible, and it leads directly to the next problem: the reliability of the system doing the watching.

VIII. The monoculture

Consider a plausible near-future evaluation. A device is built on a foundation model from one of a handful of major developers. To supplement scarce real-world data for underrepresented subgroups, the sponsor generates synthetic patient cases with a large language model. The device’s open-ended outputs are scored by an LLM adjudicator, because human adjudication of tens of thousands of outputs is expensive. After deployment, a supervisory agent built on another large model samples interactions and flags drift.

Every element of this pipeline is mentioned in the paper. Each is defensible on its own. Together, they may form a closed loop in which models of the same class generate the test, take the test, grade the test, and monitor the results.

The paper sees part of this. Question 13 asks how to keep synthetic data generated by models of the same class from reproducing the very performance gaps the evaluation is meant to detect. The concern generalizes. If the device and its evaluators share training data, architectures, and blind spots, their errors will be correlated. A case the device misunderstands is more likely to be a case the synthetic generator rendered poorly, the adjudicator scored generously, and the monitor failed to flag. The evaluation will report high agreement, and the agreement will be partly an artifact of shared assumptions.

The paper’s answer to bias in evaluation is independence, and it defines independence carefully in organizational terms. Adjudicators should be structurally independent from the sponsor and from the foundation model developer, and that requirement applies even when the adjudicator is itself an LLM. This is necessary but not sufficient. Two organizations can be legally independent and still deploy systems built on the same model. Independence also has to be epistemic.

In practice, that could mean several things. Evaluation layers should draw on different model families where models are used at all. A defined fraction of every evaluation should be anchored to human judgment and to real patient data, not as a formality but as a calibration standard for the automated layers. Sequestered test sets should be held by parties with no stake in the result, which is one of the roles the paper contemplates for independent third parties. And sponsors should be expected to estimate the correlation between device errors and evaluator errors, which is measurable, rather than assume it is zero.

There is a broader point here about concentration. If most generative devices in medicine come to rest on a small number of foundation models, a single upstream change or flaw could propagate across hundreds of products at once. Device regulation has always evaluated products one at a time. A shared dependency of this kind is a systemic risk, and it may eventually require a systemic view.

IX. The active ingredient

The paper’s proposal for voluntary Foundation Model Master Files is sensible, and the agency is candid about its weakness. Question 25 acknowledges that model developers may have limited incentive to disclose safety-relevant information. For the largest developers, medical devices are a small fraction of a very large market. A voluntary program asks them to bear disclosure costs for a modest commercial return.

The software framing may be part of the problem. Device regulation tends to treat third-party software as a component. A more useful analogy comes from the drug side of the agency, where the relationship between a finished product and an ingredient made by someone else has been worked out in considerable detail.

A foundation model is less like a software library and more like an active pharmaceutical ingredient. It determines most of the product’s behavior. It is manufactured upstream by a separate company. And small changes in its production can alter the safety and effectiveness of every product that contains it.

Pharmaceutical regulation handles this through a supply-chain framework. Manufacturers of active ingredients operate under good manufacturing practice expectations, including the ICH Q7 guidance, and are subject to inspection. They can file Drug Master Files that sponsors reference in their applications without seeing proprietary details. And FDA’s regulations require a master file holder to notify each party authorized to reference the file of any addition, change, or deletion. The finished-product sponsor remains responsible for its product, but the upstream supplier carries defined obligations of its own.

Translated to generative AI, that framework suggests what a useful Foundation Model Master File would contain beyond a model card: a commitment to notify referencing sponsors before material changes; version pinning, so that a deployed device can continue to call a fixed model; deprecation timelines long enough for re-benchmarking; and behavioral release notes describing how a new version differs on standardized evaluations, not only that it differs.

None of this requires new legal authority to begin. It can start as procurement. Device manufacturers and health systems, acting collectively, have more leverage over model developers than any one of them has alone. Contract terms covering change notification, version stability, and audit access could become standard in medical deployments well before they become regulatory expectations. That is often how regulatory science matures. The community establishes a practice, and the agency later recognizes it.

X. The thousand and first run

Return to the three experiments. The legacy measurement device gave us the same answer a thousand times, and we called it reliable without always asking whether the answer was right. The generative device gave us a distribution, and we called it a problem. The thousand radiologists also gave us a distribution, and for decades we called it the standard of care.

The discussion paper does not resolve this tension, and it does not claim to. What it does is make the tension unavoidable. Once the agency commits to judging devices against clinicians, the variability of clinicians becomes a regulatory fact, measured and documented in submissions. Once devices are authorized with greater reliance on postmarket evidence, the quality of that evidence becomes a matter of enforcement, not intention. And once a device’s function can be co-authored by its users and altered by its suppliers, the authorized object has to be defined by how it behaves rather than by what it contains.

For those who intend to respond to the agency, and many should, several commitments follow:

1. Measure reliability on human scales. Report the reliability of generative devices on the same scales used for human readers, and hold safety-critical outputs to prespecified agreement thresholds.

2. Measure the comparator as rigorously as the device. Make the choice between median clinician, expert consensus, and the counterfactual care pathway an explicit, justified decision tied to the deployment setting.

3. Define intended use as an envelope, with a measured containment rate across realistic conversations.

4. Define device identity through behavioral equivalence margins, modeled on bioequivalence and applied to any change at any layer.

5. Make the postmarket bargain enforceable from the day of authorization, with prespecified triggers and public reporting.

6. Require epistemic independence in evaluation, not only organizational independence, and estimate the correlation between device and evaluator errors.

7. Treat foundation models as active ingredients, with change notification, version pinning, and deprecation commitments established first through collective procurement and later through regulation.

There is a final irony worth naming. Medicine spent the last half century trying to make clinicians behave more like precise instruments. We wrote guidelines, built checklists, standardized endpoints, and studied practice variation as a defect to be reduced. Now we are building instruments that behave like clinicians, and the regulator is reaching for the tools we use to credential people.

The two efforts are converging on the same question:

What level of variability is acceptable in a clinical judgment, who gets to decide, and how will we know when it is exceeded?

We have never answered that question for ourselves. And the machines may not let us avoid it much longer.

Subscribe now

In February 1961, a physician at the Food and Drug Administration named Frances Oldham Kelsey came across a letter in the December issue of the British Medical Journal. It was written by a British physician, A. Leslie Florence, who had been prescribing a new sedative. Some of his patients who had taken it for long periods were reporting a painful tingling in their arms and feet.

The sedative was thalidomide. Kelsey had joined the agency only months earlier and one of her first assignments was the William S. Merrell Company’s application to sell the drug in the United States under the name Kevadon. Merrell expected approval to be a formality. The drug was already on the market in many countries. Kelsey had found the company’s evidence thin, built more on testimonials than on clinical data. The letter turned her unease into something sharper. Nerve damage, she later said, was not the kind of side effect one should expect from a simple sleeping pill. If the drug could reach the nerves of an adult, what might it do to a fetus?

Merrell pressed. Its representatives called and visited, went over her head to her superiors, and at one point described her objections as slightly libelous. Under the law at the time, an application took effect automatically if the FDA did not act on it within 60 days. So Kelsey kept declaring it incomplete, and each time the clock started over.

In late 1961, physicians in Germany and Australia linked thalidomide to a wave of infants born with severely malformed limbs. More than 10,000 children in dozens of countries would be affected. Merrell withdrew its application in 1962. That July, the Washington Post put Kelsey on its front page as the heroine of the FDA. In August, President Kennedy gave her the President’s Award for Distinguished Federal Civilian Service. In October, he signed the Kefauver-Harris Amendments, which required drugs to be proven effective as well as safe and gave the FDA authority over clinical investigations. Nearly everything we now call drug development, the INDs, the phases, the safety reports, grew out of one reviewer’s doubt.

The story is usually told as a triumph. It was also a near miss. While Kelsey was stalling the application, Merrell had been shipping thalidomide to more than 1,200 American physicians as an investigational drug, which the law then allowed with almost no oversight. Roughly 20,000 Americans received it. The FDA later identified 17 American children born with thalidomide-related defects.

More than half a century later, I was leading a team at the FDA rebuilding the process that descends directly from that episode: the review of safety reports from drugs still in clinical trials. At some point we asked a simple question. When had the agency last had a Kelsey moment? When had the FDA, looking at data from an ongoing trial, seen a danger before the company running it? We asked colleagues across the centers, people who had spent entire careers in review. It was not a formal study. But no one could name a single case.

The answer, albeit informal, was zero.

This does not mean FDA reviewers today lack Kelsey’s skill or nerve. It means the system that grew out of her doubt may be structurally incapable of producing another one. And it matters now, because the debate over the agency’s future has fixed on the wrong variable. Over the past 18 months, the FDA has lost roughly 20 percent of the staff in its major medical product centers. One side warns that fewer reviewers will mean weaker oversight. The other points out that review goals are still largely being met. Both assume that the agency’s capacity is a function of how many people it employs.

I spent years at the FDA and don’t think the FDA needs more reviewers. I think it needs to stop asking them to do work that, as designed, cannot succeed.

I. A permission to ship

Surprisingly, many people who work in drug development every day do not fully realize what the FDA’s authority over experimental drugs actually rests on. Six decades of regulations, guidance documents, user fee agreements, advisory committees, workshops, and expedited pathways have been layered on top of it. Strip those layers away and what remains is something almost mundane: the movement of goods across state lines.

The agency’s power derives from the Commerce Clause of the Constitution. Section 505(a) of the Federal Food, Drug, and Cosmetic Act of 1938 prohibits introducing a new drug into interstate commerce unless an application for it has been approved. An investigational drug, by definition, has not been approved. Section 505(i) of the Act creates the exemption that makes clinical research possible, and the Kefauver-Harris Amendments of 1962 turned that exemption into the investigational new drug system now codified at 21 CFR Part 312. Biologics follow a parallel path under Section 351 of the Public Health Service Act.

An IND, then, is narrower than most people assume. It goes into effect 30 days after the FDA receives it unless the agency places it on clinical hold. When the FDA tells a sponsor it is safe to proceed, it is, in legal terms, allowing the sponsor to ship an unapproved drug across state lines to its investigators. It is not an endorsement of the science.

This raises an obvious question: what if everything happened within one state? In theory, the Act reaches only products with a connection to interstate commerce. In practice, there is very little room in that boundary. Since the FDA Modernization Act of 1997 extended it to drugs, Section 709 of the Act (21 U.S.C. 379a) has presumed the interstate connection in any enforcement action, which puts the burden on the company to disprove it. Courts have also held that the requirement is met if any component of a product crossed state lines. In United States v. Regenerative Sciences (2014), the D.C. Circuit upheld FDA jurisdiction over a stem cell procedure performed entirely in Colorado because it used an antibiotic obtained in interstate commerce. A truly intrastate drug, made from in-state materials and tested only on in-state patients, is a legal possibility and a practical rarity. Any sponsor with national commercial ambitions will need the FDA regardless.

Within that framework, the IND stage is where most of the agency’s ongoing oversight takes place, and its focus is safety. Most INDs never become marketing applications; they die somewhere in the valley between first-in-human dosing and pivotal data. Under Section 505(i)(3) of the Act and 21 CFR 312.42, the FDA may place an IND on hold when participants would be exposed to unreasonable and significant risk, when investigators are not qualified, or when the information provided is insufficient to assess risk. For Phase 2 and Phase 3 studies, a hold may also be imposed when a protocol is clearly deficient in design to meet its stated objectives. Doubts about whether a drug will work are not, on their own, grounds for a hold. Efficacy is the sponsor’s bet to make. Safety is the regulator’s responsibility to enforce.

This is the part of the system that descends most directly from Kelsey. To understand why it has not produced a successor, we have to look at how it actually works.

II. The archive

The primary instrument of premarket safety oversight is the expedited IND safety report. Under 21 CFR 312.32, sponsors must notify the FDA within 15 calendar days of a serious and unexpected suspected adverse reaction, and within 7 calendar days when that reaction is unexpected and fatal or life-threatening.

The reports arrive in staggering numbers. From 2006 through 2014, the FDA’s Office of Hematology and Oncology Products alone received an average of 17,686 of them per year. For most of the agency’s history, and throughout my time there, they came on paper, as PDF files, and as MedWatch and CIOMS forms. They were documents, not data. A reviewer could read them one at a time. No one could aggregate them across trials, query them, or look for patterns without first extracting the information by hand.

Most of them were also noise. In 2010, the FDA issued a final rule designed specifically to reduce uninformative reporting. Five years later, we audited randomly selected expedited reports submitted to the oncology office. Only 14% were informative. More than half described adverse events that were already expected and listed in the product labeling or investigator’s brochure. We reported these findings in Clinical Cancer Research in The Majority of Expedited Investigational New Drug Safety Reports Are Uninformative.

What the agency had built, in other words, was not a surveillance system. It was an archive: a vast and growing collection of individual documents, carefully received, dutifully read, and almost impossible to see across.

III. The missing successor

It is tempting to read this as an indictment of FDA reviewers. It is not. The reviewers I worked with were skilled and conscientious and many of them would have done exactly what Kelsey did. My thesis is an indictment of the workflows.

Consider what a reviewer actually faces. Reports arrive one at a time, from one IND at a time. Each describes a single patient. A signal that only becomes visible across trials, across sponsors, or across a class of drugs is, by construction, invisible to someone reading individual documents in sequence. The sponsor holds the underlying data from its own program and will almost always see a pattern first. The FDA learns what the sponsor finds and reports.

Kelsey’s insight, it is worth remembering, did not come from data the agency could analyze either. It came from skepticism and from a letter in a medical journal. Six decades later, the agency has far more information flowing into it, and in some ways it is no better positioned to see. The premarket safety function that grew out of her doubt remains, in practice, almost entirely reactive.

This is where the staffing debate goes wrong. Adding reviewers to a document-based process adds readers. It does not add surveillance. If the current system has, as far as anyone can recall, never found a premarket signal first, then removing reviewers from it may matter less than we fear, and restoring them would matter less than we hope. The more consequential question is what the agency could do if the work itself were redesigned.

IV. What we built

We tried. INFORMED was launched in collaboration with the HHS IDEA Lab and under special federal authorities, which allowed us to bring engineers, data scientists, and entrepreneurs-in-residence into the agency to work alongside FDA physicians, statisticians, and scienstists. Its purpose was to give the FDA the organizational and technical capacity that modern data demands, as we described in Nature Reviews Drug Discovery in From big data to smart data: FDA’s INFORMED initiative and INFORMED: an incubator at the US FDA for driving innovations in data science and agile technology.

The first target was the archive. In 2016, we built a framework for the digital submission of expedited premarket safety reports, replacing paper and PDF with structured data based on the international ICH E2B standard. We published the approach in Nature Reviews Drug Discovery in Evaluating the potential for digital submission of expedited premarket safety reports to the FDA. We piloted it successfully with Merck, AstraZeneca, Novartis, and Genentech. We estimated it would save the agency at least 500 FTE hours per month, before counting the more important benefit: for the first time, the agency would have a premarket safety surrveillence system, being able to look across reports rather than only at them.

INFORMED did other things as well. It conducted the agency’s first real-world evidence studies. It developed the FDA’s first framework for AI. It worked to harmonize the agency’s data assets. It built an automated review pipeline in which submission data in CDISC format could be fed into the system and the label generated instantly. That pipeline was piloted in the Oncology Center of Excellence.

We also began large AI-enabled meta-analyses of the clinical trial data the FDA has accumulated over decades of submissions, internally with entrepreneurs-in-residence and externally through INFORMED fellowships. One example is our collaboration with MIT applying machine learning to FDA clinical trial data, published in JCO Clinical Cancer Informatics as Machine-Learning and Stochastic Tumor Growth Models for Predicting Outcomes in Patients With Advanced Non-Small-Cell Lung Cancer. We established a fellowship in AI and machine learning with Harvard and were developing other ones. The goal was a durable pipeline of technical talent inside the agency.

V. Ten years to a requirement

What happened next is instructive.

In September 2016, we published the digital submission framework in Nature Reviews Drug Discovery. Three years later, in October 2019, the FDA issued draft guidance, Providing Regulatory Submissions in Electronic Format: IND Safety Reports, describing how sponsors would submit IND safety reports as structured E2B data to the agency’s adverse event database (FAERS, since renamed the Adverse Event Monitoring System). The guidance was finalized on April 1, 2024, and the agency began accepting electronic IND safety reports on a voluntary basis that day. The requirement took effect on April 1, 2026.

Three years from a successful pilot to a draft guidance with no impact. Nearly eight to a final guidance. Almost a decade to a mandate. The enabling technology did not change materially in that time. The E2B standards existed in 2016, and our pilot was built on them.

Even now, the requirement is narrow. It covers individual case reports of serious and unexpected suspected adverse reactions, and the FDA itself notes that not all IND safety reports will go to the adverse event database. More importantly, a submission format is not a workflow. Structured data arriving at the agency is a precondition for premarket surveillance, not a substitute for it. To my knowledge, the FDA has not yet implemented aggregate, cross-IND safety analysis as a routine part of premarket review at scale. After a decade, the data is finally trickling in a form that can be seen across. Whether anyone at the agency is positioned to look is a separate question, and it remains unanswered.

VI. Piloted and shelved

Digital safety reporting is not the only example. The automated review pipeline was never implemented. After I left the agency, the INFORMED educational programs were terminated.

The pattern continues. In April 2026, Commissioner Marty Makary and Chief AI Officer Jeremy Walsh announced a real-time clinical trial initiative, with proof-of-concept studies from AstraZeneca and Amgen transmitting trial signals to the agency as they occur. The idea has a long history, including conversations Jeremy, then a contractor at the FDA, and I had years ago about why clinical trials remain so inefficient, why there is a much better way to run them, and why the FDA is well positioned to catalyze it (see “The Ship and the Signal” below). Within weeks of the announcement, Makary resigned and Walsh left the agency. The program is now, in effect, orphaned.

None of these were failures of concept. Each was piloted, and each worked. They were failures of institutionalization. The question worth asking, every time, is why.

VII. Why good ideas die at the FDA

I don’t think the answer is a lack of talent or goodwill. I think it’s structural, and it has four parts.

First, innovation at the agency depends on individuals and special authorities rather than on the institution’s operating model. INFORMED existed because of authorities granted outside the normal structure. Nothing in that structure converted a successful pilot into standard practice. When the people who championed a program leave, the program has no owner, and the default reasserts itself.

Second, the agency is measured on activity and timeliness, not on outcomes. User fee goals track whether reviews are completed on schedule. No metric asks whether the safety review process has ever detected a signal first, or how many of the reports it processes are informative. A process that meets its deadlines is treated as healthy, even if it produces little. That review timelines have reportedly held up after staff reductions of roughly 20% is consistent with this. It tells us the deadlines are being met. It doesn’t tell us what the work is accomplishing.

Third, no one owns the transition from pilot to production. Pilots live in incubators. Production lives in review divisions and IT organizations that have neither the mandate nor the budget to absorb new systems. Changing a workflow also requires coordinated change on the sponsor side, in submission standards and in guidance, and no single office is accountable for all of it. The decade between our digital safety pilot and the electronic reporting mandate is what that looks like in practice.

Fourth, the risks are asymmetric. No one is faulted for continuing a process that has been in place for decades. Adopting a new one carries visible risk for whoever signs off. Kelsey succeeded by refusing to sign. Modernization requires someone willing to sign, and the institution offers that person almost nothing but exposure. In that environment, the rational choice for any individual is to wait.

VIII. What Kelsey would need now

The staffing cuts, whatever one thinks of them, create an opening. An agency that can no longer rely on headcount has every reason to redesign the work itself. A few changes would go a long way.

Measure what the work produces. The agency should track how many premarket safety signals it identifies independently, what share of expedited reports are informative, and how much reviewer time each process consumes. What is not measured will not be defended in a budget discussion.

Use the structured safety data. The electronic reporting requirement is now in place for individual case reports. The next step is to extend it to all expedited safety reporting and to build aggregate, cross-IND analysis into the review workflow, so that a signal visible across trials can be seen by the agency, not only by the sponsor.

Give every successful pilot a decision. When a pilot meets its objectives, a named official should either move it into production by a set date or document why it will not proceed. Pilots should not be allowed to expire quietly when their champions leave.

Make the talent pipeline permanent. Fellowships and entrepreneur-in-residence programs should be part of the agency’s operating model, not experiments that end with a change in personnel.

And we should keep asking specific questions. Why did it take a decade to require structured safety data, and who is now responsible for using it? Why was the automated review pipeline never deployed? Why were the educational programs terminated? Who owns the real-time clinical trial program today?

Frances Kelsey had a single letter, a skeptical mind, and a 60-day clock she could keep resetting. Today the agency receives tens of thousands of safety reports a year, and as of this April, they may soon arrive as structured data. The raw material for her kind of vigilance has never been more abundant. What is missing is a system designed to let someone see across it.

Until we build that system, we will keep arguing about how many people to put in the room, and the next signal, when it comes, will pass through the archive the way the others have: received, filed, and noticed first by someone else.

References

Jarow JP, Casak S, Chuk M, Ehrlich LA, Khozin S. The Majority of Expedited Investigational New Drug Safety Reports Are Uninformative. Clinical Cancer Research. 2016;22(9):2111-2113.

Khozin S, Chuk M, Kim T, et al. Evaluating the potential for digital submission of expedited premarket safety reports to the FDA. Nature Reviews Drug Discovery. 2016;15:670-671.

Khozin S, Kim G, Pazdur R. From big data to smart data: FDA’s INFORMED initiative. Nature Reviews Drug Discovery. 2017.

Siah KW, Khozin S, Wong CH, Lo AW. Machine-Learning and Stochastic Tumor Growth Models for Predicting Outcomes in Patients With Advanced Non-Small-Cell Lung Cancer. JCO Clinical Cancer Informatics. 2019;3:1-11.

FDA. Adverse Event Monitoring System (AEMS) electronic submissions, including the final guidance Providing Regulatory Submissions in Electronic Format: IND Safety Reports (April 2024; requirement effective April 1, 2026).

21 U.S.C. 379a, presumption of existence of jurisdiction.

United States v. Regenerative Sciences, LLC (D.C. Cir. 2014).

University of Chicago Medicine. How a courageous physician-scientist saved the U.S. from a birth-defects catastrophe. 2020.

Discover Magazine. The Heroine of the FDA.

FDA. FDA Announces Major Steps to Implement Real-Time Clinical Trials. 2026.

Subscribe now

Some ideas arrive quietly. Others behave more like organisms. They take hold of someone, grow, and demand to be brought into the world.

The challenge is knowing which ideas deserve that kind of commitment.

In the latest episode of Precision Signals, I spoke with Peter Kolchinsky, PhD, founder and managing partner of RA Capital Management, about the people and systems that determine whether scientific ideas become medicines.

Peter’s own story begins with his parents, Soviet Jewish immigrants who came to the United States with deep faith in education, hard work, and the possibility of progress. His father was a mathematician and computer engineer who eventually started his own company. His mother trained as a mathematician, became a computer programmer, and later reinvented herself as a flower designer and entrepreneur.

From them, Peter inherited two instincts that have shaped his career: the confidence to become excellent at something and the willingness to change direction when circumstances demand it.

He initially pursued science, studying biology, conducting virology research, and earning a PhD from Harvard. But he came to recognize that his greatest contribution might not be making one breakthrough discovery himself. It might be helping other scientists turn their discoveries into medicines.

That realization led him to launch the Harvard Biotech Club, write The Entrepreneur’s Guide to a Biotech Startup at just 23, and eventually build RA Capital into one of the leading biotechnology investment firms.

Peter’s career offers a useful framework for thinking about progress in biotech. Breakthroughs do not happen in isolation. They depend on the quality of the institutions around them, the speed of the learning loops, and the willingness of leaders to engage with difficult realities.

The value of institutional memory

One of the most important themes in our conversation was the importance of remembering what the field has already learned.

Biotech is full of failure. Molecules fail. Companies fail. Clinical trials fail. Strategies that once looked promising become obsolete.

The natural response is to move on and start again. But if the lessons from those failures are not captured, the industry repeatedly pays to learn the same things.

Peter described RA Capital’s effort to build an institutional knowledge engine that maps scientific fields, competitive landscapes, emerging technologies, and failed experiments. The goal is to build an organization that becomes more intelligent with time.

The advantage of an institution is not simply that it has more information. It is that it can preserve context. What was known when a decision was made? Which assumptions proved wrong? What did the field learn from the programs that failed?

In drug development, that accumulated knowledge can be the difference between pursuing a genuinely differentiated opportunity and repeating a familiar mistake.

AI is not the same as progress

The conversation also turned to artificial intelligence and its role in drug development.

There is no question that AI has achieved remarkable engineering successes. Tools such as AlphaFold and RFdiffusion have demonstrated what is possible when sophisticated models are applied to complex biological problems.

But designing a molecule is not the same as demonstrating that it will work in people.

As Peter pointed out, drug development has one of the slowest learning loops to which AI could be applied. A company may generate a promising molecule quickly, but it can take years and many clinical programs to determine whether the technology is producing a meaningful improvement.

That creates a challenge for evaluating AI companies. If a company claims to improve the success rate of drug candidates, how many programs need to reach the clinic before that claim becomes credible? Five? Ten? Twenty-five?

The answer cannot be determined by a compelling demonstration or an attractive model output. It requires evidence generated through the long, expensive, and unforgiving process of clinical development.

This does not diminish the promise of AI. It clarifies where the promise must be tested.

The infrastructure behind innovation

Peter also offered a provocative view on China’s biotechnology ecosystem.

The discussion was not simply about scientific competition or geopolitics. It was about infrastructure.

Over the past decade, and particularly in the last five years, China has built out clinical trial capacity, medical centers, and systems capable of aggregating patient data at significant scale. That infrastructure can make it easier to identify eligible patients, enroll studies, and generate proof-of-concept data.

In Peter’s view, this may have a greater near-term effect on the pace of biomedical R&D than AI.

That observation is easy to overlook because infrastructure is less visible than a new model or platform. But clinical trial enrollment is one of the most important constraints in drug development. AI can help generate hypotheses. It cannot, by itself, enroll patients or produce clinical evidence.

The broader lesson is that innovation depends on more than discovery technology. It depends on the systems that allow ideas to be tested.

The FDA as a living institution

Peter’s perspective on the FDA has also evolved.

Early in his career, he saw the agency as a monolithic gatekeeper that could determine whether a company succeeded or failed. Over time, through direct engagement with people inside the agency, he came to see it differently.

The FDA is an institution, but it is also made up of individuals trying to make difficult decisions within a complex system.

That does not mean the agency is free from problems. Institutions can become slow, opaque, and resistant to change. They can also behave in ways that differ from the intentions of the people within them.

But treating the FDA as an unknowable force makes constructive engagement impossible.

A better approach is to build relationships, understand the reasoning behind decisions, and recognize that regulatory progress requires communication at scale.

Leadership is not a job description

Toward the end of our discussion, Peter made a distinction that stayed with me.

Many CEOs, he argued, function more like project managers than leaders. They oversee plans, manage teams, and execute against an established set of responsibilities. But leadership requires something more.

A leader should be a thought leader. Someone who understands how the field is changing, operates at the frontier of knowledge, and is willing to act when the mission demands it.

That may include doing things that are not technically part of the job description. It may mean engaging in policy debates, advocating for patients, or challenging assumptions that have become embedded in the system.

In biotech, the mission is not simply to build a company or advance a program. It is to improve the lives of patients by bringing better medicines into the world.

That mission requires scientific judgment. It requires institutional discipline. It requires the humility to learn from failure. And it requires leaders who are willing to define their role by what must be done, not by what is considered normal for their title.

The signal beneath the noise is often found in the systems, people, and decisions that determine whether progress can actually happen. That’s why I found the conversation with Peter so valuable and informative.

Subscribe now

In June 1998, Brian Druker, an oncologist and physician-scientist at Oregon Health & Science University who helped lead the development of imatinib, walked into a hospital room in Portland every thirty minutes to see whether his first patient was still all right.

The patient was a retired railroad engineer from the Oregon coast. Two years earlier, after reading Druker’s first paper on a promising leukemia compound, he had written to volunteer as the first subject. Now he was receiving the drug in a phase I trial. Every time Druker appeared at the door, the man answered in the same raspy voice: “Still here, doc. I’m okay.”

Druker was not hovering because the preclinical data had warned him to. He was hovering because of another patient years earlier.

She had colon cancer. Druker had enrolled her in a phase I trial of Taxol at close to the maximally tolerated dose. The drug caused severe diarrhea and put her in the hospital for weeks. Her tumor did not respond, and the patient’s family suggested that Druker had given them false hope. He worried they may be right.

He carried that patient into the room in Portland. What if the new drug did nothing? What if it was toxic? What if, again, the treatment caused more harm than the disease?

The engineer’s first dose was too low. Nothing happened. Druker brought him back in April 1999 at a higher dose. This time he responded, and he lived for many more years.

The landmark paper tells the story differently. At doses of 300 milligrams or more, 53 of 54 patients had their blood counts return to normal. The paper contains only the results, as it should, but not the full story: the hopes, challenges, and fears in the exam rooms and hospital wards. That erased context is where the real lessons live. That gap is why we created Precision Signals.

The published record of any major scientific advance often resembles a compression algorithm. It preserves the endpoint and discards much of what produced it: the wrong turn that opened the right path, the institutional obstacle that forced a better design, the patient who made a scientist cautious, the argument inside a regulatory agency, the capital structure that determined how long a program was allowed to fail.

One year into these conversations, the founding question has become more urgent. Biomedical information is multiplying faster than our ability to interpret it. The constraint is no longer simply producing more data. It is recognizing which data matter, what they mean, and what must happen next.

I. The seduction of a clean signal

Imatinib, later marketed as Gleevec, is one of the cleanest stories in modern medicine. It is also much messier than the version most people know.

In late 1992, Druker had spent four or five years at Dana-Farber. He had roughly twenty publications but only a few as first author. When he asked to establish his own laboratory, they looked at his record and told him his work did not seem to be going anywhere. The rejection devastated him. It also sent him to Oregon, where he could reestablish a relationship with Ciba-Geigy that had ended for his laboratory in Boston. He called Nick Lydon, the Ciba-Geigy scientist leading the company’s kinase-inhibitor work, and asked whether Ciba had compounds that inhibited ABL. Half a dozen arrived in Oregon, blinded and largely unprofiled. Three months later, one stood out: it killed chronic myeloid leukemia cells while sparing normal cells. That was late 1993.

The proposition still sounded implausible to many scientists. Biochemists doubted that an ATP-competitive inhibitor could reach a useful concentration. Oncologists doubted that a single agent could control cancer. Druker distilled the resistance into a sentence that travels well beyond medicine:

“It’s so much easier to say no than say yes.

But chronic myeloid leukemia had offered an unusually legible target. In 1960, University of Pennsylvania pathologist Peter Nowell and cytogeneticist David Hungerford identified an abnormal chromosome in CML cells. In 1973, University of Chicago cytogeneticist Janet Rowley showed that it resulted from a translocation between chromosomes 9 and 22. The rearrangement creates the BCR-ABL1 fusion gene, which encodes a constitutively active kinase that drives the disease. Druker discovered that imatinib turns that signal off.

That biological clarity shaped the trial. Druker joined leukemia specialist Moshe Talpaz at MD Anderson and oncologist-scientist Charles Sawyers at UCLA to enroll patients with CML, not a broad assortment of advanced cancers, because that was where an ABL inhibitor had a rational chance to work. They chose patients whose disease was resistant to interferon but still in the chronic phase, giving the drug enough time to reveal an effect.

The effect was extraordinary. A disease that once carried a typical life expectancy of three to five years became, for many patients, a long-term manageable condition. In the first randomized trial of newly diagnosed patients, the estimated 10-year overall survival was 83.3%. Some people from Druker’s earliest trials were still under his care twenty-six years later, and when we published our conversation, several of them found the post and shared their gratitude:

“Dr Druker is responsible for my 80th birthday next week. 26 years ago I only had a few more months to live.”

“Dr. Druker is responsible for the last 12 years of my life. Thank you so much sir. He is a real life superhero.”

“My super héros to. Thank you for saving my life dr Druker”

It is tempting to turn that triumph into a general formula: find the driver, build the inhibitor, select the patient, watch the disease recede. Much of precision oncology was built on that hope. Unfortunately, most cancers have not cooperated. Their biology is polygenic, heterogeneous, and adaptive, and our existing disease models capture only part of the system. Responses that appear decisive can be temporary, and acquired resistance to even the most potent targeted therapies today is not an exception to the treatment story; it is often the next chapter.

So the enduring lesson of imatinib is that progress accelerates when the problem has been made legible, biologically, clinically, and institutionally, and stalls when one of those forms of legibility is missing.

That distinction is why failed programs all look alike from a distance. A molecule aimed at the wrong target, a drug optimized in the wrong model, a trial built around the wrong endpoint, a technology that cannot survive a hospital workflow, and a program killed by its financing all end with the same decision: stop.

They are not the same failure. They do not teach the same lesson

II. When the signal moves

James Gulley, Chief of the Medical Oncology Service and Co-Director of the Center for Immuno-Oncology at the National Cancer Institute, was a pioneer in immuno-oncology long before the field was in vogue. Having had the privilege of working closely with him during my time at the NCI, and later across multiple projects during my tenure at the FDA, I saw firsthand how his work pierces through the fundamentals. Our collaborations spanned from the early days of immunotherapy development to more recent efforts integrating AI into the fabric of clinical research, designing adaptive clinical trials, and reimagining real-time data capture.

At every inflection point, Gulley has brought a distinctive perspective that bridges mechanistic insight with operational execution. Once it became clear that the immune system could produce durable tumor control, a harder set of questions emerged. Why does the same antibody transform one patient’s course and do nothing for another? Why have predictive biomarkers remained so unreliable? And what does resistance actually mean when a treatment acts not on the tumor itself, but through an adaptive immune system?

Even measurement becomes unstable. Conventional oncology trials observe patients at scheduled intervals, compressing continuous biology into a few visits, scans, and laboratory values. A small prospective pilot study in multiple myeloma showed what may be hidden between those snapshots: among 25 evaluable patients receiving CAR-T therapy, a wearable model detected 18 of 20 episodes of cytokine release syndrome, with a median lead time of seven hours before standard nursing recognition.

Indeed, the time of a clinical event depends on the instrument used to observe it. Change the instrument, and the event itself appears to change. This principle extends beyond physiological vitals into the subjective experience of cancer itself. Gulley’s recent work explores using machine learning to quantify pain and affect through facial recognition and voice audio waves, aiming to detect the onset of a pain crisis long before a patient self-reports it on a standard 11-point scale. When the tools of scientific inquiry become continuous and increasingly autonomous, they don’t just capture data differently; they redefine the clinical reality we are able to treat.

Of course, challenges and uncertainties do not disappear when immunotherapy works. It moves into the thyroid, the heart, the nervous system, and other organs that were never the intended target. Afreen Shariff is the endocrinologist who directs Duke’s Onco-Endocrinology Program and sees such patients. Jon McDunn is a biomedical scientist and president of the cancer-research nonprofit Project Data Sphere. In our conversation on immune-related adverse events, another form of compression was exposed. A trial report can name a toxicity and assign it a grade. A clinician facing a symptomatic patient needs to know which clue matters, who should see the patient, and how quickly. Shariff has built an e-consult model around the two gaps she sees most often: expert triage and access to that expertise.

In our conversation on Precision Signals, Sushil Patel, chief executive of the cancer-therapy company Replimune, reflected on the future of immuno-oncology and the fraught regulatory path of their lead asset. We spoke during a critical window of uncertainty: after the FDA issued a complete response letter in July 2025, but before a second one arrived in April 2026.

The core of the dispute centered on what a single-arm study could actually prove. The FDA’s April letter argued that the application could not isolate the drug’s contribution when combined with nivolumab, raising concerns about the study population and response assessment. Replimune countered that the observed responses were durable and clinically meaningful for patients who had run out of alternatives.

Then, the narrative shifted rapidly. The FDA accepted a resubmission in June. On July 30, an advisory committee voted 10 to 3 that the IGNYTE trial results were indeed evaluable and clinically meaningful. Just a week later, on August 6, the agency granted accelerated approval to Tudriqev with nivolumab for adults with unresectable advanced cutaneous melanoma whose disease had progressed on a PD-1-based regimen.

The data told a complex story. IGNYTE enrolled 140 patients. For the 91 patients with at least one non-injected lesion—the efficacy-evaluable population used for approval; the objective response rate was 24.2%, and the median duration of response was 14.1 months. Continued approval now depends on confirming clinical benefit in the ongoing randomized phase 3 IGNYTE-3 trial.

The important point here is not that one side was right all along. It is that the exact same development program can support sharply different judgments about bias, attribution, urgency, and acceptable uncertainty.

III. Machines relocating uncertainty

Olivier Elemento directs Weill Cornell Medicine’s Englander Institute for Precision Medicine; oncologist Cora Sternberg is its clinical director. In the inaugural episode, we discussed why so many medical AI tools perform well in retrospective studies and so few earn a durable place in clinical practice.

The distance between technical validation and adoption is not explained by clinician skepticism alone. Trust has to be built in the setting where the tool will be used, against a relevant comparison, with an outcome a clinician can recognize. Data access, workflow, prospective testing, reimbursement, and accountability often determine more than the model architecture does.

In her episode of Precision Signals, Michelle Longmire, a physician-scientist who co-founded and leads the clinical-trial technology company Medable, described encountering this exact bottleneck from the operational side: biology is advancing far faster than the apparatus built to test it. Study startup is painfully slow, sites are scarce, data remains fragmented, and participation is often punishing for patients. At Medable, she set a deliberately destabilizing target known as the 1:1:1 model: one day to start a study, one day to enroll a participant, and one year to complete it. A goal inside the existing framework invites incremental optimization. A goal outside of it forces the entire model to be questioned.

Thomas Clozel, an oncologist who co-founded and leads the biomedical AI company Owkin, left clinical practice because biology, as he put it, had become too complicated for the unaided human mind. That realization led Owkin to build systems designed to learn across institutions without requiring patient-level data to ever leave them.

What that kind of computation can reveal is perfectly illustrated by MesoNet. Trained on nearly 3,000 digitized pathology slides from the French MESOBANK collection and independently validated on a cohort from The Cancer Genome Atlas, the model predicted survival in malignant mesothelioma more accurately than conventional histology. But the most striking finding was not its accuracy; it was its focus. The regions carrying the strongest prognostic signal were located primarily in the stroma (areas characterized by inflammation, cellular diversity, and vacuolization) rather than in the malignant cells themselves.

For generations, pathologists were trained to focus their attention on malignant cells. The model had no such inherited bias. It found a vital pattern in the tissue that the human eye had learned to look right past.

That is the profound promise of computation in medicine, and also its inherent discomfort. A model can discover a signal without possessing the conceptual vocabulary that makes it intelligible. It can tell us exactly where to look before we understand why looking there matters. The machine has not eliminated uncertainty—it has simply handed the uncertainty back to us in a much more interesting form.

David Fajgenbaum, a physician-scientist at the University of Pennsylvania and co-founder of the drug-repurposing nonprofit Every Cure, reached computation through a different route: his own near-death, using data architecture to save his own life.

As a third-year medical student, he developed idiopathic multicentric Castleman disease and suffered repeated episodes of multiorgan failure, nearly dying five times. With no reliable roadmap, he banked and analyzed his own blood across relapse and recovery. The data implicated the mTOR pathway and led him to sirolimus, an immunosuppressant used in transplantation that had never been used for his disease. He has now been in remission for more than a decade.

He could have treated that outcome as a miracle of individual persistence. Instead, he treated it as a systems problem. He co-founded Every Cure on the premise that useful therapies may already exist, separated from the patients who need them by the way biomedical knowledge is organized.

Its MATRIX project attempts to score approved drugs against human diseases at scale. ARPA-H now lists an award of up to $124 million for the program, with a second phase intended to advance at least 30 repurposing opportunities into preclinical or clinical validation.

Druker arrived at the same principle by accident. After imatinib’s approval, a dermatologist in Utah called to ask about trying a quarter-dose in patients with hypereosinophilic syndrome. Druker thought the idea was implausible. The disease “melted away.” Only later did researchers identify a PDGF-receptor rearrangement that explained why those patients were exquisitely sensitive to the drug.

The empiricism came first. The mechanism caught up.

IV. What survives is a design choice

The CEO Roundtable on Cancer began with a fundamental judgment about institutions. In 2001, President George H.W. Bush asked Robert A. Ingram, then chief executive of GlaxoWellcome, to convene leaders from business, government, academia, and the nonprofit sector to do something “bold and venturesome” about cancer. That challenge led to the CEO Cancer Gold Standard and, eventually, to Project Data Sphere, an open-data and AI model development organization launched in 2014.

The core premise behind these initiatives was that structures determine behavior. If an existing system cannot produce the needed behavior, exhortation is not enough. The structure itself has to change.

That idea recurred continuously throughout the first year of Precision Signals. We saw it in how evidence is disseminated, how discovery is incentivized, and how capital is deployed.

Consider the architecture of medical consensus. Clifford Hudis, an oncologist and the chief executive of the American Society of Clinical Oncology (ASCO), explained how ASCO had to structurally adapt to clinical evidence that now evolves faster than traditional publishing cycles. In 2022, the organization launched living guidelines for stage IV non-small-cell lung cancer, maintained by standing expert panels and updated continuously as practice-changing data emerge, ensuring that bureaucratic lag does not delay patient care.

In the same way that ASCO is redesigning the flow of evidence, others are redesigning the flow of discovery and capital. Karen Knudsen, a cancer biologist and chief executive of the Parker Institute for Cancer Immunotherapy, detailed how the organization was built on a novel intellectual-property model designed to seamlessly push discoveries between academic centers, nonprofits, and industry. The system assumes ambitious science will frequently fail, but it rewards collaboration over silos and recycles its returns into new research.

Similarly, Dan McHugh, an investor at the oncology investment firm Yosemite, broke down how the structure of capital dictates the clock of innovation. Yosemite combines conventional venture equity with grant capital that does not take IP. Because these two pools of money tolerate different forms of uncertainty and carry different timelines, they change the fundamental math of drug development, dictating how much evidence a program must produce, how quickly it must produce it, and which failures it is allowed to survive.

Yet, every structural choice ultimately lands on a human being. Sunil Verma, AstraZeneca’s global head of oncology, brought the argument back to the exam room. Molecular profiling may identify the exact therapy matched to a tumor, but it cannot decide the risks a person is willing to bear, the physical function they most want to preserve, or what they need to be able to do a year from now. Precision is molecular. Personalization is human.

Mike Krzyzewski (Coach K), the legendary Duke men’s basketball coach, understood that distinction long before he had the clinical language for it. His father operated an elevator at Willoughby Tower in Chicago; his mother cleaned floors at the Chicago Athletic Club at night. What they gave him, he noted on the podcast, was not wealth, but the certainty that someone stood behind him. He never had to believe that failure would leave him alone.

When the conversation naturally shifted to teams, his philosophy translated perfectly to biomedicine. A care team is not just the physician and the nurse. It is the technician, the trial coordinator, and the person who cleans the room. A functioning team makes every contribution visible and every person an owner of the outcome. “Talent,” he reminded us, “makes talent better.”

Coach K then shared the story of Jim Valvano, the former North Carolina State basketball coach who transitioned from his greatest rival to his closest friend. While visiting Valvano during the final months of his life, Krzyzewski remembered Valvano learning a staggering statistic: only one in six cancer researchers seeking support actually received funding. Valvano’s response wasn’t statistical; it was desperately human: What if one of the other five had the cure for his cancer?

The V Foundation, created by Valvano and ESPN in 1993, has since awarded nearly $458 million in cancer research grants, using an endowment to cover administrative expenses so that 100 percent of direct donations fund the science.

Valvano’s question gets to the heart of the matter. The unfunded five are not a metaphor. They are the human consequence of a selection system. Capital sits upstream of experiments, experiments sit upstream of evidence, and evidence sits upstream of patients. The tolerance for risk built into the very first decision eventually, inevitably, appears in the last.

V. Following the signal

Biology, technology, evidence, capital, policy, regulation, and care do not wait politely for one another. They interact continuously. A choice made in one domain silently rewrites what is possible in the next.

Imatinib reached Brian Druker because a career rejection pushed him across the country and reopened a dormant industry relationship. A wearable algorithm detected a clinical event hours before the hospital because it monitored continuously rather than periodically. A pathology model found a vital prognostic signal in the exact tissue that experts had been trained to discount. A transplant drug halted a rare inflammatory disease because a medical student became his own longitudinal study. An oncolytic virus pivoted from two FDA rejections to an accelerated approval because the data were argued over, reanalyzed, and judged under the pressure of an urgent unmet need.

This is exactly what polished, retrospective narratives tend to remove: contingency. They make progress look inevitable in hindsight. It never is.

Moving into its second year, Precision Signals will continue hunting for the moments when theory collides with reality and when persistence pushes us into new terrains of discovery. If there is a conversation we need to have, please comment below with suggestions. To follow the journey, please consider subscribing on YouTube.

The noise will only keep multiplying. The task is to find the signal and build a system capable of acting on it.

Subscribe now

Sometime in the middle of the nineteenth century, Rudolf Virchow looked at blood under his microscope and found cells that should not have been there. He gave the disease a name that was, in the manner of the age, a direct translation of what he saw. Leukämie. White blood. The naming was not an act of understanding. It was an act of description elevated to the status of a category. And for a hundred years afterward, an entire field built its taxonomies on Virchow’s original visual habit.

This is how the grammar of medicine has always worked. We look. We name. We reify the name until the name becomes the thing.

Thomas Clozel, MD, an oncologist and the co-founder and CEO of Owkin, has spent the last decade trying to write a new grammar for biology, the deep structure of how disease is described, categorized, and understood at all.

This is the substance of the latest episode of Precision Signals.

Chemistry is a confined problem. Biology is an open one.

Thomas is the son of two French physicians who left the clinic to build their own biotech and put several drugs on the market. His grandmother was one of the first women physicists in France. Science was the first language he learned. Medicine was the second. Coding, which he took up under Dr. Olivier Elemento at Weill Cornell Medicine during a research year, was the third.

His first rotation at Mount Sinai was under Dr. Valentin Fuster, the great cardiologist. What Fuster taught him, more than any clinical technique, was the discipline of I don’t know. Three words, Fuster insisted, that separate the good doctor from the excellent one. Three words that any serious reasoning model will also, one day, have to learn how to say.

In late 2016, with his co-founder Gilles Wainrib, a machine learning researcher trained at École Normale Supérieure, Thomas founded Owkin. The contrarian bet was that chemistry was a confined space. It had rules: Molecules bind, and proteins fold. Given enough compute, an AI could, in principle, learn chemistry the way it had learned to play Go. Biology was different, and it was where causality lived. The question was not how to design a better molecule. The question was what disease actually is.

The mesothelioma paper

To do this work, Owkin needed patient data at a scale no lab could provide alone. Its first bet was on federated learning. Rather than pulling data out of hospitals, Owkin built infrastructure that let its models travel to the data and train there, in situ, without ever taking the data out. Hospitals in Germany, France, the UK, and the US signed on. The engineering was hard. The trust was harder.

In 2019, Owkin published a study on mesothelioma histopathology that, to me, was the first real signal that something was different. Most digital pathology work at the time was trying to replicate the taxonomies human pathologists had already invented. The AI was being asked to imitate the human eye. Owkin’s mesothelioma model was asked to do something else. It was asked to look at the slides without a taxonomy and to organize what it saw. What it produced was a set of categories no human had ever named, corresponding on later analysis to distinct survival curves. Disease entities that had been sitting in front of pathologists for a century without ever being recognized as such.

This is the real grammar of biology and a demonstration that the units of disease are not fixed.

The AI scientist

Over the last two years, Owkin has consolidated its tools, its models, and its data assets into a single architecture called K Pro.

K Pro is not a wrapper on a general-purpose model but a reasoning system trained on real patient data, orchestrated through workflows that Owkin builds by having Claude write code for it, and validated through Owkin’s own wet lab and clinical partnerships. Thomas describes it as an AI scientist that is beginning to have ideas of its own. He tests it in two modes. In research mode, he asks it to find a new target or a new molecule. In clinical mode, he asks it what to do for a fourth-line bladder cancer patient. He says the answers, when checked against his network of experts, have begun to surprise him.

Success

I asked Thomas what would need to happen for him to say, someday, that Owkin had succeeded. He gave the answer I already knew he would give. It will only count if he treats something important for the world. Not the billion-dollar valuation, their approved diagnostics, or their programs in the clinic. Only a treatment that changes a patient’s life.

Everything else is preamble.

Subscribe now

I. The phone call

Late in 2015, someone inside FDA picked up the phone and called Pfizer.

The request was simple. The oncology division wanted whatever evidence the company possessed on crizotinib in patients with ROS1-positive non-small cell lung cancer. There was no large randomized trial waiting in the wings, no thousand-patient database still being cleaned, no elaborate statistical exercise. Pfizer sent what it had.

It amounted to fifty patients.

The evidence came from a single-arm study led by Alice Shaw at Massachusetts General Hospital, published in the New England Journal of Medicine the previous year. Independent review found an objective response rate of 66%, slightly lower than the investigators had reported. The National Comprehensive Cancer Network had already incorporated crizotinib into its treatment guidelines for ROS1-positive disease. By the time the supplemental application reached FDA, the clinical community had, in effect, already rendered its verdict.

In March 2016, FDA approved crizotinib for metastatic ROS1-positive non-small cell lung cancer. Priority review had compressed the timeline to roughly six months. The entire evidentiary package could have fit inside a modest summary table.

Those of us working at FDA felt something difficult to admit publicly. We had not led the field. We had arrived after it.

What even seasoned industry veterans tend to misunderstand about FDA, and about most regulatory authorities of its kind, is what such agencies actually do. They do not manufacture clinical conviction. They do not, in the first instance, decide what constitutes evidence or what counts as benefit. Those judgments are made elsewhere: in laboratories and clinics, in academic departments and professional societies, in trials and meta-analyses and the slow accumulation of shared experience. By the time a regulator acts, the community has usually already reached its conclusion. What FDA does, at its best, is ratify it.

II. The last reader

FDA occupies a singular place in modern medicine, but not quite the one people imagine.

It is tempting to think of the agency as the author of clinical standards, the institution that decides what constitutes evidence and, by extension, what becomes accepted practice. Its decisions undoubtedly shape medicine. Patients gain or lose access to therapies because of them. Companies rise and fall on them. Yet FDA’s confidence almost never emerges in isolation. More often, it crystallizes only after the scientific community has spent years generating, debating, and refining the evidence.

In that sense, FDA is less the author than the final reader.

The pattern repeats so consistently that, once recognized, it becomes difficult to ignore. Companion diagnostics, biomarker-driven approvals, surrogate endpoints, molecular classifications, imaging criteria: nearly all entered regulatory practice only after clinicians, investigators, statisticians, and professional societies had begun converging on a shared understanding of what the evidence meant.

RECIST is perhaps the clearest example.

The Response Evaluation Criteria in Solid Tumors was not written by FDA. It emerged in 2000 from an international collaboration among the European Organisation for Research and Treatment of Cancer, the US National Cancer Institute, and the National Cancer Institute of Canada. FDA adopted RECIST because the clinical trial community had adopted it and, imperfect as it was, nothing better existed.

Twenty-five years later, RECIST still anchors nearly every regulatory decision based on tumor-based endpoints in solid tumors, its limitations well documented and widely acknowledged. In a meta-analysis I conducted at FDA involving roughly 13,677 paired radiologic assessments, radiologists disagreed on response category about 30% of the time, and over 40% for hard-to-measure tumors. Investigators know this. Sponsors know it. FDA knows it. And yet RECIST persists, because no alternative has accumulated enough evidence to command broader consensus.

Regulators do not abandon standards simply because they are imperfect. They abandon them when the community offers something demonstrably better.

III. The lipstick and the scan

When I took the results of that meta-analysis to one of the senior officials at the FDA, he read the numbers without much surprise. Then he looked up.

“You know, Sean, I never liked RECIST.”

He said it the way you say something you have said to yourself, quietly, for years.

“Do you know what I trust the most?”

“What?”

“The lipstick sign.”

He told me about a patient he had once treated, years earlier. A woman with metastatic colorectal cancer had arrived for treatment frail, cachectic, and visibly declining. After several cycles of chemotherapy she returned transformed. She had regained weight. Color had returned to her face. She walked into clinic wearing bright red lipstick.

“I really didn’t need to see the scan,” he said. “I knew the treatment was working. We would continue therapy.”

That kind of observation still carries real evidentiary weight in everyday practice. Almost no oncologists outside of clinical trials use RECIST. Radiology reports in the metastatic setting tend toward a softer, descriptive vernacular: this lesion is smaller, that one stable, correlate clinically. Clinical benefit is something experienced before it is measured.

Inside the trial world, that era has largely disappeared.

Today the scan is expected to settle the question. RECIST measurements, lesion diameters, independent central review, progression-free survival curves, objective response rates: these have become the language through which benefit is translated into evidence. The patient still matters, of course, but the patient’s improvement must now survive quantification before it can influence a regulatory decision.

FDA did not invent that transformation. The community did.

Once the community accepted RECIST-defined imaging as the common language of efficacy, the agency adopted the same language because it had become the only one everyone understood. The senior official had spent his career watching that shift. And still, when asked what he trusted, he named the lipstick.

That was the first time I realized that the FDA does not manufacture consensus. It waits for a consensus it can ratify.

IV. What FDA clearance cannot buy

None of this is confined to the drug world of the last generation. The same dynamic is now playing out around clinical AI, and playing out badly.

Nearly every ambitious clinical AI company in oncology has organized itself around FDA. Get the clearance. Attend the workshops. Cultivate the reviewers. The unstated assumption behind all of this is that FDA is the gatekeeper whose approval unlocks the market.

If the argument of the preceding sections holds, that assumption is inverted. FDA is not the gatekeeper. It is the last reader, waiting for the community to arrive at a conclusion the agency can ratify. A tool built to persuade FDA before it has persuaded anyone else is a tool built for the wrong step in the sequence.

The clearest illustration of the alternative comes not from clinical AI but from a molecular diagnostic developed a generation earlier. The specific route it took is no longer available to most software today, but the strategic logic behind it still is.

Consider Oncotype DX. Its developer, Genomic Health, was founded in 2000 and eventually acquired by Exact Sciences in 2019 for approximately $2.8 billion. The company built a 21-gene recurrence score assay for early-stage, hormone receptor-positive breast cancer, and made a deliberate choice not to seek FDA clearance. It ran the test as a laboratory-developed test out of a single CLIA-certified laboratory, and directed its evidence at oncologists and payers rather than regulators. The American Society of Clinical Oncology and NCCN incorporated the assay into their guidelines. Medicare covered it. TAILORx and RxPONDER, both prospective, established clinical utility. FDA never approved Oncotype DX. It never had to. The moat has held for more than twenty years.

That exact route is closed to most clinical AI. Software as a medical device making substantive clinical claims cannot route around FDA the way Oncotype DX did as a laboratory-developed test (LDT). The 2024 attempt by FDA to bring LDTs under device regulation was vacated in March 2025 by the Eastern District of Texas, and the agency formally rescinded the rule in September 2025. That ruling does not extend to software. SaMD is a device. It falls squarely within the Federal Food, Drug, and Cosmetic Act. The only narrow exit is the Clinical Decision Support carveout in the 21st Century Cures Act, and most substantive clinical AI cannot fit through that door, unless one is developing free software that clinicians will likely never use.

But the strategy behind Oncotype DX still applies. Strong evidence is universal currency. It compels agreement wherever it lands, from oncologists, from guideline committees, from payers, and yes, from FDA. The mistake most clinical AI companies make is not that they have prioritized FDA in the wrong order. It is that they have treated FDA as the audience whose blessing is the point of the exercise. FDA is one reader among several. Depending on the product, it may not be the most consequential. For an LDT, the agency is not the decision-maker at all. For a SaMD, it matters, but no more than a guideline committee or a payer. Build a tool that oncologists actually need. Generate the evidence to prove it. The rest, FDA included, will follow.

What most clinical AI companies do instead is pursue 510(k) clearance and hope that a demonstration of substantial equivalence to some predicate device, often something that predates the modern imaging or oncology workflow by a decade, will function as a proxy for clinical validation. It does not. Physicians do not treat 510(k) clearance as evidence. Payers do not reimburse against it. The ten-thousandth 510(k)-cleared stethoscope is fine. A 510(k)-cleared oncology AI tool making substantive claims about patient management is not fine, because the pathway was never designed to answer the question the market is actually asking.

There is a related misapprehension that keeps consulting rosters full. It is the belief that access to former FDA officials, however senior, can substitute for the evidence the market is asking for. Companies pay handsomely for such counsel, hoping that some hidden regulatory knowledge, some subtle understanding of how the agency really thinks, will unlock a path their peers have not found.

It does not exist.

Ex-FDA leaders can offer real value. They can explain established standards. They can help navigate submission logistics. They can flag procedural risks outsiders would miss. What they cannot do is dispense a formula for how to redefine a paradigm, or how to push a novel biomarker or endpoint through the agency. Not because they are withholding it. Because it is not theirs to withhold. Paradigms are not redefined at FDA. They are redefined by the community that generates the evidence. FDA, in due course, follows.

The question the market is asking is: does this tool change what I should do for my patient? That question can only be answered by prospective data. It cannot be answered by regulatory clearance of any kind, except a PMA with prospective (often randomized) data, which is the kind of evidence that will move not only the regulator but the community at large, including payers and clinicians.

V. The contract that kills

This raises an obvious question. If prospective evidence is the only real path, why do so few AI companies take it?

The answer is culture and worldview.

The implicit contract most clinical AI companies signed with their investors is a software contract. Fast iteration. Low marginal cost. Quick path to revenue. SaaS multiples. That contract is structurally incompatible with the kind of evidence generation the clinical AI products actually require. You cannot run a prospective validation study in a diagnostic imaging AI on a two-year software burn rate. The math does not work.

This is why so many companies in the space end up chasing 510(k) shortcuts and FDA relationship-building. It is not that some founders do not understand what real validation looks like. It is that they raised on a timeline that does not permit it. The shortcut is the only path available given the capital structure, and the shortcut leads to a product physicians will not adopt and payers will not reimburse.

The honest version of the pitch to investors is this. Clinical AI making substantive claims is not a software business. It is a diagnostics business with a software component. The comparables are not Epic or Veeva. The comparables are Genomic Health. Foundation Medicine, taken private by Roche in 2018 at an enterprise value of roughly $5.3 billion. Guardant Health, still public and worth several billion dollars. Each of those companies spent years pre-revenue or sub-scale, generating clinical evidence, before the market gave them the returns they eventually earned. None of them cut corners on evidence. All of them built moats many of their competitors still cannot cross.

An investor who does not understand this should not be in the round. A founder who does not raise from investors who understand this will be forced into the shortcut, and the shortcut will kill the company.

VI. The evidence is the moat

For the founders and investors who do accept the trade, the upside is enormous.

Category ownership in diagnostics generates margins and durability that most software businesses envy. Once a tool is in NCCN, once payers reimburse at scale, once the clinical evidence has been written into guidelines, the incumbent’s advantage becomes structural. It is not the algorithm. It is not the code. It is not the clearance. It is the evidence. And the evidence takes years, hundreds of millions of dollars in some cases, and prospective trials that a competitor cannot shortcut. That is the true barrier to entry, and it is one a 510(k) can never create.

There is a second-order effect that first-movers capture and fast-followers cannot. The company that runs the first prospective trial in a category also gets to help define what the endpoints in that category should be. It gets to shape the consensus FDA will eventually align with. That is not merely a business advantage. It is a form of scientific authorship, and it accrues to whoever is willing to fund it.

But scientific authorship, ultimately, is not the work of any single company. It is the work of a community. Investigators. Physicians. Statisticians. Professional societies. Guideline committees and sponsors. All converging on shared standards for what evidence should look like. Sometimes that means defending the standards we have. Often it means replacing them with better ones.

When clinical AI approaches FDA with bespoke solutions that carry no peer validation, with half-formed proposals that lack credibility among the stakeholders, the agency cannot act unilaterally to sanction them. In a way, it should not. Manufacturing consensus is not what regulators are for. That responsibility belongs to us. Building conviction strong enough, and universal enough, that the data speaks for itself is not FDA’s job.

Which brings us back to fifty patients.

Fifty patients was never the story.

Fifty patients was simply the point at which the story no longer required additional chapters.

By the time FDA approved crizotinib, the agency was not discovering the truth. It was acknowledging one the clinical community had already accepted.

Every successful regulated medical product eventually reaches that moment. The question is never whether the evidence satisfies a regulator but whether the evidence has become so persuasive that every serious reader arrives at the same conclusion. When that happens, guideline committees converge, payers follow, physicians change practice, competitors fall behind, and FDA’s decision becomes almost inevitable.

The moat is not the algorithm. It is not the regulatory clearance. It is not even the product.

The moat is the evidence.

Everything else follows from that.

And the evidence is ours to build.

Subscribe now

Success, whether in science, business, or investing, is ultimately an exercise in signal processing: recognizing meaningful patterns, filtering out noise, and connecting ideas that initially appear unrelated.

Few people embody that way of thinking better than Sahir Ali, founder and managing partner of Modi Ventures, my guest on the latest episode of Precision Signals. His career spans engineering, quantitative finance, cancer research, entrepreneurship, and venture capital. Those experiences ultimately shaped BioStack, Modi Ventures’ investment strategy for identifying technologies that create value across the life sciences. Rather than evaluating companies within traditional sector boundaries, BioStack seeks foundational technologies whose impact compounds across multiple layers of the innovation ecosystem.

One story that stayed with me was Sahir’s early investment in the hardware startup Groq. At the time, many questioned why a life sciences-focused investor would back an AI infrastructure company. Through the lens of BioStack, however, the investment was never about chips. It was about recognizing foundational technologies that would eventually reshape biology, drug discovery, and medicine. That conviction proved remarkably prescient, culminating in Nvidia’s landmark multi-billion-dollar transaction centered on Groq’s technology and talent. Long before AI infrastructure became one of the most sought-after assets in technology, BioStack had identified its strategic importance.

The conversation is ultimately less about venture capital than it is about intellectual discipline. How do you distinguish signal from noise? How do you build conviction before consensus forms? How do you recognize that the technologies most likely to transform medicine may emerge from places that, today, seem entirely unrelated to healthcare?

Those questions matter far beyond investing. They matter to anyone trying to understand where science, technology, and society are headed next.

I hope you’ll enjoy the conversation as much as I did.

Subscribe now

I. The two clinics

In the summer of 1865, the Hungarian obstetrician Ignaz Semmelweis was committed to an asylum in Vienna under circumstances that remain disputed and grim. Within two weeks he was dead, at forty-seven, of “blood poisoning,” an infection of nearly the same family as the one he had spent his life trying to prevent. The man who discovered how to stop women from dying of childbed fever died of sepsis, in confinement, largely disbelieved.

Two decades earlier he had been a junior physician at the Vienna General Hospital, which ran two maternity clinics side by side, the first staffed by physicians and medical students, the second by midwives. Women were admitted essentially at random, by the day of the week, and yet the two clinics produced wildly different outcomes. In the first, maternal mortality from puerperal fever hovered near one in ten, and in bad months worse; in the second, a few corridors away, it ran closer to one in twenty-five. The women knew, and they begged for the midwives’ clinic. Some preferred to give birth in the street.

Semmelweis worked the difference like an engineer, ruling out climate, crowding, delivery position, the route of the priest, every fashionable explanation in turn. What remained was the thing no one wanted to name: the physicians, unlike the midwives, performed autopsies and then walked from the dissection room to the delivery ward with nothing more than a cursory rinse. He ordered handwashing in chlorinated lime, and mortality in the first clinic collapsed toward one or two percent within months.

The data was overwhelming and the intervention nearly free, and still it did not spread, not for years, because accepting it meant accepting something intolerable: that the doctors were the vector, that the healers carried the disease from one bed to the next on their own hands. Being right was not enough. Against institutional inertia, it almost never is.

An institution adopts a change only when the change becomes legible in the terms it already uses to keep score. I have come to think this is the single most important thing to understand about any institution, the FDA at this moment included, and it is also what the current coverage least understands. The visible story is disruption; the consequential one is quieter, the conversion of a useful idea into an institutional obligation, the moment a thing that can be done becomes a thing the agency is responsible for doing.

II. One slide

The FDA is now narrated through rupture: reductions in force, reorganizations, senior departures, the accumulating evidence of an agency under strain. Read enough of it, and the conclusion begins to feel foreordained: that the agency is being hollowed out and alarm is the only honest posture. Some of that alarm is warranted, because a public health agency is not an abstraction, and its missteps can eventually reach the bedside.

But there is another account, harder to see because it leaves fewer public traces, and I want to make it from the inside, from the years I spent there, because that is where I learned how reform actually moves through a scientific bureaucracy: slowly, unevenly, and only when a working idea is translated into a form the institution can own.

Some years ago I gave an hour-long talk at the National Academies on the use of artificial intelligence in premarket safety surveillance, and for a subject that invites grand claims I had prepared none. I had one slide for a one-hour talk, a photograph of a computer monitor fringed on every side with yellow sticky notes.

This, I told the room, is how premarket safety is actually done at the FDA. There is no surveillance database behind it. The seven-day and fifteen-day expedited safety reports, the ones a sponsor is legally bound to file when something serious, suspected, and unexpected happens to a patient on an investigational drug, arrive as PDFs or by fax, and are read by a person and remembered, or not, on paper pressed to the edge of a screen. Before we can have a serious conversation about AI, I said, we have to get to data, just data, because the ability to draw a bar graph would already be a profound change.

The room laughed, because the line sounded like exaggeration. It was not.

Stay with that fact and it stops looking like inefficiency. At the investigational stage, before a drug is approved for anything, safety is the FDA’s central statutory authority; efficacy is the sponsor’s burden to prove later, at the NDA or BLA stage, if the program survives the valley of death where most INDs vanish. Safety is the lever the law hands the agency over a drug in human testing, the single hook it is empowered to act on, and behind that lever, for all of the agency’s modern history, sits no real surveillance at all. Sticky notes on a monitor, notes on paper, a spreadsheet someone started and never finished, a reviewer turning to the reviewer at the next desk to ask whether they had seen the same event and whether it meant anything: institutional memory carried in conversation, which is to say carried nowhere durable. What is remarkable is how well the reviewers did anyway, holding in trained judgment what the system gave them no instrument to hold. The deficit was never theirs.

Worse, the machine produced volume without proportionate knowledge. The oncology office was receiving thousands of expedited reports a year, and in an audit we published, we drew a random sample of 160 and read each against the law’s three-part test: serious, unexpected, a suspected reaction to the drug. Twenty-two qualified, fourteen percent; the other 86% were individual case reports that, stripped of any comparator or denominator, could neither raise a true alarm nor retire a false one. The system was at once overloaded and under-informed, activity in abundance and knowledge in deficit.

This is one of the things INFORMED, the data science incubator I founded at the FDA, set out to repair. We built the other version of premarket safety, structured and machine-readable and near real-time, a system that treated a safety report as a row in a dataset rather than a page to be filed and half-remembered, and it worked, not on a slide but on real reports, in conditions close enough to production that the result was a demonstration and not a promise.

Then I made the mistake some institutions teach you to stop making. I brought people at the FDA a number.

At full implementation, by our estimate, structured digital safety reporting would return to the agency at least 500 full-time-equivalent hours a month, five hundred hours of labor lifted off transcription and handed back to judgment. I thought the number would do the work for us, since it was, after all, large enough to seem self-evident.

It did very little.

It took our team an embarrassingly long time to understand why, and that understanding is among the most useful things I carried out of the building. Efficiency is not the FDA’s native currency. No one’s mandate there is measured in hours saved, and no career is made or ended by labor returned to the system or capital reallocated to taxpayers. The agency’s currency is caution, scientific defensibility, and above all the avoidance of the catastrophic miss, the approval that should never have happened, and I had walked in with a strong answer to a question almost no one in the room was paid, promoted, blamed, or remembered for asking.

We had brought Semmelweis’s basin to a room that had not yet been shown what was on its hands. It was not a failure of evidence, nor a failure of the people in it, but a failure of translation: we had shown the system worked in a pilot with large sponsors, but we had not yet made it legible as something the agency was responsible for owning.

III. Where good systems go to wait

A pilot can live on ingenuity, charisma, borrowed staff, and the temporary suspension of normal rules; infrastructure cannot. The moment a pilot must become infrastructure, it needs three things an incubator cannot grant: a permanent owner, a budget line that survives the next reorganization, and a mandate that crosses the boundary between offices with rival incentives. An innovation that belongs to everyone in general belongs to no one in particular, and that is where good ideas go to wait, not in a laboratory but in a gap in the org chart, the institutional form of forgetting.

We built several things that went there to wait: the structured safety system from that slide; the early scaffolding for an expedited IND process, meant to compress the dead time between an investigator’s readiness and an active application; real-world evidence pipelines; biosensor work, the first attempts to gather evidence closer to the patient and farther from the spreadsheet; AI infrastructure, in the pre-LLM era; the harmonization of the agency’s data into a single system. Each one worked, and some stalled the instant their champions left, for reasons that had nothing to do with technical merit and everything to do with custody.

This is why the present moment is more interesting than the headlines allow.

IV. Surgery on the living

Against the grain of the political temperament, I have begun to feel something close to optimism, and the reasons are not atmospheric but concrete: decisions, made by named people, in the last two years, to turn orphaned capabilities into assigned responsibilities.

Start with digital safety reporting, finally being implemented, not because the technology suddenly matured, since it never needed to, but because the only thing ever missing was at last supplied: someone with the authority, willing to spend their political capital, decided to own it and make it the standard across the agency. Once you know to look for that kind of decision, the pattern surfaces everywhere.

The agency’s leadership has lately been dismantling barriers that stood for years. In December 2025, it removed one of the oldest, issuing guidance that lets sponsors submit real-world evidence without first stripping and re-identifying patient data, a technical change with large consequences for the clinical registries and claims databases that had been sitting just out of reach. At nearly the same moment, and for the first time, the agency began to publish an honest accounting of what real-world evidence is actually entering its review pipeline, and the numbers, though modest and easy to over-read, point in a direction that is not in doubt. Protocols submitting real-world data to the drug center roughly tripled in a single year, from 11 to 31, and sponsors are beginning to file studies that use such evidence not merely to watch a drug after approval but to help establish that it works in the first place. Externally controlled designs, all but absent the year before, have started to appear, the approval packages of the decade’s end taking shape now, and the running tally of approvals informed at least in part by real-world evidence has climbed past a dozen.

ISTAND is the same story in a different register, the qualification pathway the agency built for tools that fit through no existing door: digital measures, wearables, AI methods, in silico models of human organs. Its first in silico submission, an AI model of the liver that predicts whether a drug candidate is likely to cause injury, was accepted not long ago, and this matters more than it sounds, because the reason real-world evidence submissions still show almost no contribution from digital health technologies is not that the sensors fail but that there has never been a qualified road from a wrist to a regulatory decision. ISTAND is that road, finally being paved.

An expedited IND initiative has resurfaced too, no longer an incubator sketch but a formal pilot, a rolling submission platform meant to shorten the path from drug identification to first-in-human study and to minimize the clinical holds that stall it. And there is the real-time clinical trial initiative, which I wrote about in The Ship and the Signal, that can reduce the latency between data and the regulator while leaving every safeguard in place.

None of this is disruption for its own sake; it is something harder, closer to surgery on a beating heart, the target moving under the hands while the stakes refuse to pause. That is the work underway at the FDA.

V. The bar that does not move

The agency today is working to modernize its substrate, the speed and structure with which evidence moves, the latency with which a signal is seen, the durability of the record on which judgment rests, while the judgment itself must remain as demanding as it ever was. The good version of these reforms makes the agency faster at being careful, not faster at being permissive.

Even those who concede the agency needs transformation will warn, reasonably, against throwing the baby out with the bathwater, and they are right that there is a baby. The agency’s scientific judgment, its institutional caution, its hard memory of every drug that looked good and was not, is precious and very nearly irreplaceable, and a careless hand could pour it out in a single season and fail to rebuild it in a generation.

But the idiom conceals its central premise, that the bathwater is inert, unpleasant perhaps but harmless in itself, and Semmelweis is the standing refutation, because in his wards the medium was not neutral. The contamination was the thing hurting the patients. The legacy substrate at the FDA, the fax and the PDF and the 86% of reports that are noise, is not merely old; it is a hazard to the very people the agency exists to protect, because a signal seen too late is not an administrative delay but potentially a patient harmed. To guard the bathwater in the name of the baby is a category error, and inaction is not prudence but a quieter way of choosing the greater risk.

The task was never to choose between the baby and the bathwater. It was to change the water without losing the child, and that is the work ahead, and the discipline it asks for: not razing the ward, not denying the fever, but operating on the living.

So I hold both halves of the glass, which is the only honest way to hold it. The disruption is real, and some of it may leave scars; the reinvention is also real, and a meaningful share of it is precisely the kind the agency needed and could not deliver on its own, for structural reasons that had little to do with money or talent. The question is whether the reinvention becomes durable before the disruption becomes the whole story.

VI. How an institution knows

Semmelweis had a system that worked, and the hard part was getting the institution to fold it into its own self-description: this is how we know, this is how we remember, this is how we will notice when reality begins to contradict us. That is the deepest meaning of modernization, not the substitution of software for judgment but the modernization of the way institutions know. A regulator that receives evidence as paper can deliberate only at the speed and fidelity of paper; a regulator that holds evidence as memory can forget; a regulator that receives evidence as data can still be wrong, but it can be wrong in ways that are visible, contestable, and correctable.

Semmelweis’s basin was never only a basin. It converted a moral and scientific fact into a routine, something a hospital performed every day without having to be persuaded again. The new instruments arriving today; AI, sensors, real-world data, real-time trials, earn their place only if they do the same work: forcing the institution to see its own contact with evidence more clearly, what it knows and what it is missing, where delay has become risk, where caution has curdled into concealment, where noise has been mistaken for vigilance.

Half full is the only honest reading, not because the glass is full, nor because the problem is solved, but because the agency has begun to take ownership of the instruments through which it sees the world. Semmelweis could not save himself, and was disbelieved to the end, yet the practice he could not make them accept outlived the person who showed them the stain, and that is the asymmetry worth betting on and the reason to do the work.

In public health, as in medicine, repair begins not with the claim that the hands are clean, but with the discipline that makes unclean hands impossible to ignore.

Subscribe now

I had the honor of speaking with Mike Krzyzewski, known to most of the world simply as Coach K, for a new episode of Precision Signals.

There are many ways to introduce Coach K. You can start with the national championships, the Olympic gold medals, the decades at Duke, or the generations of players and coaches whose lives he helped shape. But what stayed with me most from this conversation was not the résumé. It was the way he thinks about people.

Coach K has spent his life studying how individuals become a team. Not in the shallow sense of putting talented people in the same room, but in the deeper sense of building the trust, discipline, and shared purpose that allow people to do difficult things together.

That question has obvious relevance to sports. It also has deep relevance to medicine.

In biomedicine, we often talk about progress through the things we can name: a new therapy, a sharper biomarker, a better trial, a more powerful model, a new algorithm. Those things matter but none of them becomes meaningful in isolation. A discovery becomes progress only when people carry it across the distance between an idea and a patient’s life.

That distance is scientific, but it is also human.

It depends on whether researchers can ask better questions, whether clinicians can interpret evidence with judgment, whether regulators can make decisions under uncertainty, whether institutions can work across boundaries, and whether patients can trust the people and systems around them. The movement from discovery to impact is not just a technical process. It is a human one.

That’s why I wanted to have this conversation with Coach K.

His philosophy of leadership began long before Duke, in Chicago, with parents who taught him about work, faith, family, and the confidence that comes from knowing someone believes in you. It continued at West Point and in the Army, where leadership was not an abstract concept but a responsibility carried in front of other people. It matured at Duke and with Team USA, where talent was never the whole story. The real work was creating the conditions in which talent could become trust, and trust could become performance under pressure.

We also talked about his work in cancer advocacy and his connection to the CEO Roundtable on Cancer. That history matters to me because cancer is not a problem any one institution can solve alone. It requires science, prevention, clinical research, data sharing, regulatory judgment, patient advocacy, and care delivery to move together in ways that are difficult but necessary.

The same theme came up when we discussed artificial intelligence in medicine. Coach K does not approach AI as a technologist, which is part of what made his perspective so valuable. He approaches it as someone who understands time, attention, and trust. If AI can reduce the burden that keeps clinicians staring at screens instead of patients, if it can give doctors more space to listen, explain, and be fully present, then it can serve something profoundly human.

That is the version of AI in medicine I care about most. Not AI as spectacle. Not AI as a substitute for judgment, empathy, or responsibility. But AI as a way to remove the friction that keeps people from doing the work only people can do.

And yes, because I’m a New Yorker, I asked him about the Knicks.

His answer began with Madison Square Garden and the force of a fan base that had found its voice again, but it returned to the same idea that runs through the entire conversation: belief. What happens when people feel part of something together. What changes when a team, a city, an institution, or a community begins to trust again.

That is what I kept coming back to after the conversation ended.

Basketball, medicine, cancer, technology, leadership. These may seem like separate worlds, but they share a common problem. Talent is not enough. Technology is not enough. Ambition is not enough. The real question is whether people can build the trust and discipline required to turn individual effort into collective progress.

That is the human work behind progress.

I’m grateful to Coach K for joining Precision Signals and for a conversation that moved across basketball, medicine, AI, cancer advocacy, and the deeper question of how we do hard things together.

Subscribe now

I. Chicago, May 31

Like many of my colleagues, I was in the room for the phase 3 results of RASolute 302, a randomized trial testing daraxonrasib, a first-in-class oral inhibitor of active RAS, against standard chemotherapy in patients with previously treated metastatic pancreatic cancer.

The presentation was a late-breaking plenary at ASCO’s 2026 annual meeting at McCormick Place in Chicago. When the Kaplan-Meier curves appeared on the screen, the two survival lines pulling apart with an unusually wide and stable separation, the audience stood and applauded before Brian Wolpin, director of the Hale Family Center for Pancreatic Cancer Research at Dana-Farber and lead investigator of the trial, had finished speaking.

It lasted 42 seconds.

When the applause subsided, Wolpin noted that the time had not been built into his talk.

These results deserve to be understood on their clinical merits, which are substantial. But they also deserve to be understood as the product of a particular kind of scientific thinking, one that required forty years of accumulated failure before we could ask the right question.

II. The disease that has resisted almost everything

Pancreatic ductal adenocarcinoma kills most of the people it touches. In 2026, roughly 67,500 Americans will be diagnosed, more than half with metastatic disease at presentation. For those patients, the five-year survival rate is around 3%. When first-line chemotherapy stops working (and it almost always does), second-line treatment has historically offered a median overall survival of 6 to 7 months and a median progression-free survival of 3 to 4 months, at the cost of substantial toxicity.

One of the central molecular drivers has been known for decades. Oncogenic RAS mutations are present in more than 90% of pancreatic tumors, most involving substitutions at KRAS codon 12. These mutations lock RAS in a GTP-bound active state, driving proliferative signaling the cell cannot turn off. The therapeutic conclusion has always seemed obvious: inhibit active RAS, and you could reach nearly every patient.

For forty years, nobody could.

III. The geometry of failure

Gilbert Ryle, the British philosopher, introduced the concept of a category mistake in 1949 to describe what happens when someone searches for a thing in a place where, by the nature of the category being used, it cannot exist. His canonical example is a visitor touring Oxford who walks past every college, every library, every building, and then asks: but where is the University? She had seen the University. She failed to recognize it because she was looking for a different kind of thing. The error was conceptual before it was empirical.

The forty-year failure to inhibit RAS-GTP was, at its root, a category mistake of this kind.

Drug discovery has long operated within what is known as a substance ontology: a framework in which the fundamental objects of interest are independent entities that bear properties. Proteins have pockets. Targets have binding sites. A protein is druggable or it is not. RAS-GTP presented exactly the kind of problem this framework was least equipped to solve: a smooth, largely featureless surface, no deep binding pocket, and picomolar affinity for its natural substrate GTP, making competitive displacement essentially impossible. We treated druggability as an intrinsic property of the protein itself, found no such property in active RAS, and concluded it could not be done. So we looked downstream.

In 2013, Kevan Shokat, professor of cellular and molecular pharmacology at UCSF and a co-founder of Revolution Medicines, cracked this open by identifying a transient pocket on the KRAS G12C mutant. That insight led to sotorasib and adagrasib, validated RAS as a pharmacological target, and changed the conversation. But these drugs bind KRAS in its GDP-bound inactive conformation; they reach RAS only when it is already off. And G12C is rare in pancreatic cancer. For the patients who most needed RAS inhibition, the first generation was, by design, largely beside the point.

The central question remained: how do you inhibit active RAS, GTP-bound and constitutively firing?

IV. What nature knew, and what we failed to generalize

The breakthrough that produced daraxonrasib did not come from finding a hidden pocket on RAS. It came from escaping the assumption that druggability had to be a property of RAS itself. The decisive question was relational: what if the drug did not have to bind RAS alone?

This required abandoning an assumption so deeply embedded in traditional drug discovery that most of us had stopped noticing it as an assumption. One target, one protein, one binding site. The paradigm is powerful and ubiquitous. For active RAS, it was a dead end.

What is striking in retrospect is that nature had been demonstrating a different paradigm for decades, and we had been watching. Rapamycin does not inhibit mTOR by occupying a pre-existing mTOR pocket; it first binds FKBP12, and the resulting complex engages mTOR through an interface neither partner could access alone. Cyclosporin A recruits cyclophilin A to inhibit calcineurin. Thalidomide forces cereblon to bind and degrade transcription factors it would otherwise ignore. These are among the most consequential drugs in medicine, and they share a common logic: a small molecule conscripts an endogenous protein to create a new interaction that blocks a biological process indirectly. Before active RAS could be drugged, nature had already shown the way.

But none of these drugs emerged from a rational design program. Rapamycin is a natural product isolated from a soil bacterium found on Easter Island in 1972. Cyclosporin A came from a Norwegian fungus identified in a Sandoz screening program. Thalidomide was a sedative whose mechanism in cancer was reverse-engineered decades after it was withdrawn from the market for causing birth defects. In each case the sequence was the same: discover the molecule, observe what it does, spend years understanding why. These drugs did not begin as a design principle. They ended up revealing one, and we largely filed the lesson away. Each mechanism was admired as elegant and categorized under its own domain (immunophilin pharmacology, cereblon biology) without asking whether the underlying logic could be deliberately applied to problems that had resisted direct inhibition.

This is a recognizable pattern in science. Paradigm-shifting evidence accumulates in plain sight for years before someone asks whether it generalizes. The data were there. The inference was not.

Shokat’s group made that inference in 2019, publishing proof-of-concept work showing that bifunctional ligands could recruit cyclophilin A to GTP-bound RAS and block its interaction with B-Raf. This was not yet a drug. But it was the answer to a question we had not been asking.

V. Sanglifehrin A: the handle that made it possible

Revolution Medicines, founded in 2014 by Martin Burke, a chemist and natural product synthesis pioneer at the University of Illinois, Michael Fischbach, a microbiologist and biosynthesis expert at Stanford, and Shokat himself, had an explicit founding thesis: natural products could be reverse-engineered into precision medicines for targets that conventional drug discovery could not reach. The company now had the concept. It needed the chemistry.

The scaffold came from a natural product discovered by Novartis in 1999 and never developed as a drug: sanglifehrin A, a bacterial macrocycle with high affinity for cyclophilin A. Its value was structural. Its binding mode leaves a solvent-exposed face pointing outward, a chemical vector free to be linked to a RAS-binding pharmacophore. One face buried in cyclophilin A, the other engineered toward RAS-GTP.

The team built sanglifehrin-inspired bifunctional molecules and solved the crystal structures of the resulting tricomplexes with KRASG12C. What those structures revealed could not have been predicted from studying either protein in isolation. The drug sits in a composite pocket that is partly cyclophilin A and partly RAS: a neomorphic interface that does not exist when either protein is present without the other. The molecule remodels cyclophilin A’s surface to create a high-affinity interface with active KRASG12C. The pocket was not discovered. It was created.

VI. From G12C to every RAS: the making of daraxonrasib

The first clinical molecules from this platform, elironrasib for G12C and zoldonrasib for G12D, were mutation-selective, exploiting covalent chemistry anchored to specific mutant residues. Elegant, but insufficient for pancreatic cancer, where the full spectrum of oncogenic RAS variants collectively accounts for nearly the entire patient population.

Daraxonrasib was built around a broader premise: engineer a molecule that engages the conserved switch regions shared across all oncogenic RAS variants rather than the residues that distinguish one mutation from another. The result is a reversible, noncovalent molecular glue: macrocyclic, orally bioavailable, mutation-agnostic.

The mechanism is worth holding precisely. Daraxonrasib binds cyclophilin A first, forming a binary complex that then engages RAS-GTP across the composite interface. The tricomplex blocks downstream effectors (RAF, PI3K) from reaching the RAS surface. The drug does not fit into RAS. It fits into a relationship between two proteins, a relationship it calls into being. For G12X mutants there is an additional effect: daraxonrasib can stimulate GTP hydrolysis, actively accelerating RAS back toward its inactive state rather than merely blocking its downstream signals.

That distinction matters. Daraxonrasib is not a better RAS inhibitor in the conventional sense. It is a different way of thinking about what an inhibitor can be.

VII. What 13.2 months means

RASolute 302 enrolled 500 patients across 59 sites in six countries, randomized 1:1 between daraxonrasib 300 mg once daily and investigator’s choice of chemotherapy. Results were published in the New England Journal of Medicine the same day they were presented at ASCO.

Of enrolled patients, 91.8% had RAS G12 mutations, the primary endpoint population. In that population, median overall survival was 13.2 months with daraxonrasib versus 6.6 months with chemotherapy, a hazard ratio of 0.40 representing a 60% reduction in the risk of death. At 12 months, 53.3% of daraxonrasib patients were alive versus 18.7% on chemotherapy. Median PFS was 7.3 versus 3.5 months. Objective response rate was 33.2% versus 11.8%.

The context that makes 13.2 months remarkable: the landmark first-line regimens in this disease (FOLFIRINOX, gemcitabine plus nab-paclitaxel, NALIRIFOX) achieved median overall survival of 8.5 to 11.1 months in previously untreated patients. Daraxonrasib, in patients who had already exhausted those regimens, compares favorably with all of them.

Beyond survival, time to pain deterioration was 9.0 versus 3.7 months. Patients were not only living longer. They were feeling better longer. Treatment-related discontinuation occurred in 1.2% of daraxonrasib patients versus 11.2% on chemotherapy. Hematologic toxicities and peripheral neuropathy, the defining burdens of chemotherapy in this disease, were largely absent.

FDA expanded access was authorized in May 2026. The broader question remains: how quickly can a system built around sequential evidence review deliver a result this large to the patients who need it?

VIII. A question worth sitting with

There is something in this story worth dissecting.

When Wolpin showed those survival curves at ASCO, the knowledge that daraxonrasib halves mortality in metastatic pancreatic cancer became public. But that knowledge existed at the data cutoff in February. It existed, in effect, from the moment the trial was powered enough to see it. And yet the patients sitting in oncology clinics that same week, the ones not enrolled in RASolute 302 and not navigating expanded access, were receiving second-line chemotherapy with a median survival of under seven months, because the drug that could double that had not yet cleared the regulatory process required to reach them.

This is the gap we need to talk more about. Not a failure of any single institution, not a problem unique to daraxonrasib, but a structural feature of how we move evidence from signal to treatment. The trial completed enrollment in November 2025. The data cutoff was February 2026. Results were presented publicly in June. Submission, review, approval, and prescribing infrastructure still lie ahead. Each step is defensible in isolation. Together, they add up to months; and in pancreatic cancer, months are the unit of survival.

The architecture we built to evaluate evidence was designed for an era of slower science, slower information, and limited computational capacity. It made sense then. The question worth asking now is whether a system built around collecting data, freezing it, submitting it, and reviewing it as a completed artifact is still the right model when the signal is a hazard ratio of 0.40 on a pre-specified primary endpoint in a 500-patient randomized trial, and the disease being treated measures survival in single-digit months.

IX. The regulatory paradigm needs its own conceptual leap

The daraxonrasib story is about changing the unit of analysis. We could not inhibit active RAS because we were looking for a single-protein binding site that did not exist. The breakthrough came when someone changed the question from “where is the pocket?” to “what relationship could we create?” The same logic applies to how we move evidence from trial to patient.

The current model treats drug evaluation as a sequential, bounded process: conduct the trial, freeze the data, submit a dossier, review it, decide. I called this the HMS Salisbury model in an earlier essay, “The ship and the signal”: the idea that medical evidence must be collected, transported, and unpacked before it can inform a decision, the way James Lind’s citrus observations had to cross the Atlantic before anyone acted on them. It was the right architecture for its time. It is increasingly misaligned with a world of molecularly targeted therapies, large and unambiguous signals, and diseases that measure survival in single-digit months.

The FDA’s first real-time clinical trial pilot (AstraZeneca’s TRAVERSE study in mantle cell lymphoma and Amgen’s STREAM-SCLC study in limited-stage small cell lung cancer) proves the model can change. In that framework, endpoint and safety signals reach reviewers as they emerge rather than arriving frozen in a dossier after trial completion. It is not yet a transformation. It is a proof of concept, and a sign that a fundamentally different paradigm is within reach.

X. What the standing ovation was really for

I have been to many ASCO plenary sessions. I have not seen many moments like the one on Sunday.

I think people standing in that hall were not applauding a hazard ratio. They were applauding the end of a forty-year impasse in a disease where failure had become the baseline expectation, and where the same grim prognosis had been delivered to patients year after year despite everything the field had tried.

But the full measure of what Sunday means depends on what we do next.

The scientific paradigm changed when someone asked a different question about where a binding site could be found. The regulatory paradigm needs the same move: away from evidence as a frozen artifact reviewed after the fact, toward evidence as a continuous signal evaluated in real time. The science and the system have to advance together, or the science advances into a bottleneck.

It took forty years of failure, a philosopher’s insight about category mistakes, three natural products that nobody designed, a bacterial macrocycle that sat unused for twenty-five years, and a company founded on the belief that nature encodes answers to questions we have not yet learned to ask. The regulatory infrastructure capable of delivering the resulting therapy to patients without avoidable delay does not fully exist yet either.

Creating it will require the same thing that created daraxonrasib: the willingness to question whether the current frame is the right one.

The question changed. That is why the room stood up.

Now the room has to go to work.

Subscribe now

Every era of medicine settles on its own metaphor, and ours has chosen precision.

The word is everywhere now. Precision oncology. Precision diagnostics. Precision medicine. It suggests a field that has learned to aim, a field that has moved beyond the blunt instruments of the past and into a more exact relationship with disease. But precision in oncology is not a destination. It is a discipline: the patient work of asking what has gone wrong in a cell, whether that wrongness is essential to the cancer, and whether the answer might be small enough to hold in a single molecule.

Dr. Brian Druker built his career inside that question.

In the latest episode of Precision Signals, I traced the arc of his life and the arc of the field he helped invent. The conversation kept returning to a simple observation: the questions he sat with thirty years ago, about targets, mechanisms, and the courage to test an unfashionable idea in humans, are the questions oncology has not finished answering.

Brian grew up in St. Paul, the youngest of four children of first-generation immigrants from Eastern Europe. His father was a chemist who held patents on printing processes used by 3M, and the expectation that one of the children would become a doctor lived more in the atmosphere than in explicit instruction. As a boy, Brian wanted to play baseball. Science, it turned out, had other plans for him.

He landed at UC San Diego for undergrad, where he worked in John Abelson’s lab purifying restriction enzymes. He stayed for medical school, then went to Barnes Hospital in St. Louis for residency, which he still describes as one of the richest learning experiences of his life.

Somewhere along the way, he wrote in a class paper that the field would only make tangible progress by understanding what distinguishes a cancer cell from a normal cell. It was the kind of sentence a student writes before knowing how long a life can be organized around a single premise. It became the thesis statement for everything that followed.

That premise eventually led him to chronic myeloid leukemia. CML was an unusual cancer in one crucial respect: its biology had a visible center. The Philadelphia chromosome produced the BCR-ABL fusion kinase, and the disease depended on that aberrant signal. If one could inhibit ABL without damaging normal cells, the logic was clean. The question was whether such a drug could exist.

The imatinib story is now told as if inevitability was hiding inside it. It was not.

Within weeks of arriving at OHSU in 1993, Brian called Nick Lydon at Ciba-Geigy and asked whether the company had anything that might inhibit ABL. Lydon said they did. Within three months, Brian had data showing he could kill leukemia cells without harming normal ones. That sentence now sounds like the beginning of a revolution. At the time, it sounded implausible enough that both Nature and Science declined the foundational paper.

The skepticism was not irrational. The dominant view held that any ATP-competitive inhibitor would shut down too many kinases in the body. Even if the biology worked, the market appeared small: perhaps 5,000 CML patients a year. For a large company, this did not look like the kind of opportunity around which one reorganized a development program.

It took five years to convince Novartis to enter clinical trials. By 1997, Brian brought his data directly to a toxicologist at the FDA. The first human trial began in 1998, restricted to Philadelphia chromosome positive patients. His first patient was a train conductor from the Oregon coast. The leukemia went into remission.

Within six months, 53 of 54 patients had responded.

Gleevec was approved in May 2001 by the FDA. It landed on the cover of Time. The life expectancy of patients with CML moved from three to five years to an 89 percent five-year survival. A disease that had carried the emotional architecture of fatalism became, for many patients, a chronic condition. The molecule then extended into gastrointestinal stromal tumors once KIT came into view, offering the first hint that a single targeted agent might cross disease boundaries when the biology was shared.

This is the part of the story the field remembers. It is also the part some have sometimes misunderstood.

Imatinib did not prove that every cancer has a single switch waiting to be turned off. It proved that, in the right disease, with the right dependency, the right molecule can rearrange the future. CML was unusually legible. The target was not merely present. It was central.

The hundred kinase inhibitors that followed imatinib produced cycles of enthusiasm and disappointment partly because oncology internalized the wrong lesson. The field treated the imatinib paradigm at the time as a template when it was also a special case. It tried to export the form without always having the underlying biology.

That distinction matters.

In most tumors, the biology is distributed, redundant, adaptive, and heterogeneous. A single lesion rarely explains the whole disease. The cancer cell sits inside tissue, immune pressure, metabolic context, and time. This is why Brian’s argument today for bolder combination strategies carries weight. Combination therapy is easy to praise and hard to do. It multiplies toxicity, development complexity, regulatory uncertainty, and commercial coordination. But some diseases will not yield to the elegance of a single molecule because their biology is not organized around a single point of failure. In that setting, precision cannot mean one target, one drug, one patient. It has to mean understanding enough of the system to intervene at more than one essential point.

The same humility should govern how we talk about genomic data. Oncology has generated a vast amount of it. The clinical gain has been real, but not proportional to the scale of sequencing. A mutation list is not a disease model. A pathway diagram is not an intervention strategy. A biomarker is not a mechanism unless the biology makes it one.

This is where the conversation turned naturally to AI.

Brian was candid about what AI can and cannot do for biology. He drew on a Jensen Huang anecdote to make a point that stayed with me: physics has fixed laws; biology does not. AI is most useful where the underlying rules are at least partly knowable. In biology, the rules are contingent, context-dependent, and often hidden behind layers of measurement error and incomplete observation.

AI will matter enormously. It will find patterns, generate hypotheses, design molecules, read images, organize literature, and compress work that used to consume whole careers. But AI cannot rescue a field from not understanding the biology it is asking the model to optimize. I always emphasize that when the training data captures proxies, the model learns proxies. If the endpoints are noisy, the model learns noise with confidence. If the disease mechanism is only partially understood, the model may accelerate motion without improving direction.

The next quarter century in oncology may be defined less by better targets or more data than by the harder work of understanding biology well enough that our tools, AI included, finally have rules to follow.

The human texture of my conversation with Brain was quite obvious near the end, in a story about a letter Brian recently received from a patient named Kevin in Jakarta. Kevin had been newly diagnosed with CML. He wrote to thank Brian for the medicine that allowed him to plan a future with his children.

That is the standard that matters.

There is a final grace note in Brian’s story. In the late 1990s, Brian was interviewed by an AP reporter who had grown accustomed to press releases announcing cancer cures that never materialized. Her notes captured a careful skepticism: nice guy, good with his patients for a researcher, but that drug is not going anywhere.

That reporter is now his wife.

It is a funny ending, but also a useful one because the skepticism was not foolish. Unfortunately, most cancer-cure stories do fail. Most early discovery breakthroughs do not survive contact with patients. I think we need skeptics because patients need us to distinguish hope from evidence.

What made imatinib different was not that it avoided skepticism but that it survived it. And that may be the best definition of precision oncology we have:

Following a biological idea far enough, and carefully enough, that it can meet a patient and still be true.

Subscribe now

In nine days, thousands of oncologists will converge on McCormick Place in Chicago for the ASCO Annual Meeting. As it does every year, the field will perform its ritual of compression: a year of progress distilled into five days of plenaries, posters, hallway conversations, late dinners, and hurried meetings between sessions.

The person responsible for the machinery behind that ritual is Dr. Clifford Hudis, ASCO’s CEO, and my guest on the latest episode of Precision Signals.

Cliff grew up in Northeast Philadelphia in the 1960s in what he describes as a modest, middle-class household. His father worked continuously from the age of eighteen. His mother went back to school and became a public school teacher. There were no physicians in the family. The early scaffolding came from a cousin training to be an allergist, who served as an informal mentor, and from a summer in eighth grade spent in a laboratory at Hahnemann, grinding up earthworm muscle for someone else’s physiology experiments.

The public schools of that era were investing seriously in math and science. His parents, recognizing something in him, took out a loan to send him to a small Quaker school where AP biology and chemistry classes had only three or four students. From there he entered a six-year accelerated BA/MD program at Lehigh and the Medical College of Pennsylvania and graduated from medical school at twenty.

His internal medicine residency in Philadelphia brought him into oncology through a clinic full of grateful patients and still-incomplete science. In the mid-1980s, he moved to New York and was running the emergency room at Mount Sinai, where the oncology service operated as its own department under James Holland. It was there that Cliff first encountered Dr. Larry Norton from a distance.

Memorial Sloan Kettering had already taken root in his imagination years earlier through an unlikely source: Brian’s Song, the James Caan film in which Chicago Bears running back Brian Piccolo dies of a midline germ cell tumor at Memorial. In 1988, Cliff arrived at MSK as a fellow. That same fall, Larry Norton was recruited from Mount Sinai to launch the breast cancer service.

The team’s first home was not a gleaming cancer center floor. It was a two-bedroom apartment next to the hospital, with Cliff wedged into a corner of the living room.

What followed was an extraordinary decade. G-CSF. Paclitaxel. Trastuzumab. BRCA testing. Aromatase inhibitors. Capecitabine. In roughly ten years, breast cancer treatment changed in ways that still shape practice today. US breast cancer mortality peaked around 1990 and has since fallen by roughly 40%, driven in large part by systemic therapy advances that emerged from that era.

Cliff became Chief of the Breast Service in 1998, succeeding Larry Norton. But throughout our conversation, he is careful to reject any heroic version of the story. He describes himself as a collaborator, one person helping translate a much larger collective enterprise into clinical practice.

His path to ASCO also ran partly through Larry. Norton’s deep involvement with the cooperative groups, especially CALGB, pulled Cliff beyond the walls of Memorial and into the broader oncology community. Cliff eventually won a seat on the ASCO board as treasurer, a position that was directly elected at the time. He spent three years receiving what he calls an informal one-on-one MBA in how a professional society actually works.

He later stood for ASCO president, and his term coincided with the society’s fiftieth anniversary. The theme he chose was Science and Society, a phrase that now feels strikingly prescient. It reflected a decade of flat NIH and NCI funding, amounting to roughly a 25% real-dollar cut after inflation, and the early erosion of public trust in scientific methods. During that period, he found himself knocking on doors in the House and Senate, asking in one form or another when the United States had decided to cede scientific leadership.

When Allen Lichter stepped down after a decade as ASCO’s CEO, Cliff put himself forward. He inherited an organization of five hundred people, sixteen direct reports, and no fully integrated strategic plan. Over the next eighteen months, he reorganized it carefully. He is characteristically humble about the transition. There was, as he puts it, no universe in which he was qualified on his own to run ASCO. But the institution was strong enough to tolerate a new chief executive and bring him along.

On AI, Cliff offers an analogy I have kept thinking about. The future AI-powered electronic medical record, he suggests, may function like the lane-keeping and adaptive cruise control systems already built into ordinary cars. Clinicians will still feel autonomous when they are practicing within the lane, but the system will gently nudge them when they begin to drift.

That view is reflected in ASCO’s collaboration with Google Cloud, which produced a constrained Gemini chatbot that reads only from ASCO material, runs with its creativity dialed to zero, and highlights sources for every sentence it returns. Cliff suspects that the chatbot itself may look quaint in a decade, the way a journal devoted to indoor plumbing would look quaint today. The more durable shift is toward living guidelines, refreshed on an approximately eight-week cadence.

There is an implication here that Cliff does not overstate, but the listener can feel it. If oncology knowledge becomes truly continuous, the ASCO Annual Meeting of 2036 will not look like the Annual Meeting of 2026. The ritual of compression may give way to something more fluid, more continuous, and perhaps more closely aligned with how science now moves.

Cliff closes with a line he borrowed from Albert Bourla about visionary leadership and competent skepticism. It is the right note to carry into McCormick Place next week.

Subscribe now

0. Pre-conditions: three things that must hold

This essay is about projections, and every projection of this kind smuggles in assumptions. And more than me, it is the assumptions that do the actual work. In writing this essay, I have smuggled in three assumptions, pre-conditions that need to be met. The timelines I outline only hold if they hold. If they fail, what follows may not be slower or harder. It may be a different essay entirely.

My first assumption is that the instruments we use to measure clinical benefit continue to be repaired in plain sight. Tumor response by RECIST in oncology. Depression scales in psychiatry. Six-minute walk distance in cardiology. These are not facts about the world by any means. They are conventions we trained ourselves to agree on, and in the case of RECIST, independent expert readers disagreed on whether a patient had progressed in roughly 30% of more than 13,000 paired assessments in an FDA-led analysis. An AI trained on those endpoints does not transmute their noise; it inherits it, and reproduces it at scale, with more confidence than is warranted. My first precondition is that novel endpoint development becomes regulatory-grade work, with appropriate clinical validation, publishable noise floors, and audit trails, and is not run within firms competing on the same flawed reference frame.

The second assumption is that human data accumulates the way infrastructure accumulates, not the way leftovers accumulate. Longitudinal, multimodal, reuse-consented data on actual patients across actual disease courses is the binding input. It is the thing models still cannot generate on their own. If the field treats it as the residue of clinical trials, never to be shared for secondary use research, the field will keep training on poor proxies and call them “ground truth.” There are no ground truths and no gold standards, only better approximations. If the field treats data liquidity as deliberate infrastructure, with consent frameworks that hold across decades and storage that survives platform turnover, the next ten years are productive. The fork is happening now, and it is not technical. It is contractual and architectural.

My third assumption is that the public trust, which permits clinical research at all, does not collapse during the transition. Trust is older than every tool stacked on top of it, and it is replaceable in the way load-bearing walls are, which is to say, slowly and with care. A trust failure in this decade, whether through a high-profile AI-generated mistake affecting a patient, or through opacity in how synthetic controls and digital endpoints are incorporated into evidence packages, can cost the field a decade of optionality. The precondition is that the people running the most ambitious programs are also the people most attentive to the slow social work of explaining, disclosing, and refusing to oversell.

AI capex is not AI capability. Saying so out loud, repeatedly, while building, is part of the build.

If my three assumptions hold, the timeline that follows is achievable. If they do not, the failure mode is the one we recognize: another decade in which capability claims outrun the scaffolding built to validate them. The argument that follows is conditional on the scaffolding being repaired in time.

I. Where AI stalls in biomedicine

AI is decisively powerful wherever a problem is bounded and the space of possible answers is already densely mapped. This is why it has functionally solved protein structure prediction, why it is compressing molecular design, and why retrieval and summarization in regulatory and pharmacovigilance work are no longer hypothetical. The problem space is finite. The data is dense. The verification loop is short.

A blunter way to put this: most of what we currently call AI in drug discovery is engineering, not biology. Protein structure prediction, molecular design, pattern recognition in pathology and radiology, and retrieval and summarization across regulatory documents are engineering tasks performed on dense priors with short verification loops. They are real work, and they are real progress, and the speed they introduce is genuine. They are not, however, the same activity as understanding why a disease behaves the way it does in a body. Conflating the two is the move that produces the cycle in which capability claims outrun the scaffolding.

AI is strikingly less useful wherever the question is open and the human data is thin, which is most of what determines whether a drug actually works inside a body. The biology of disease, the heterogeneity of response, the interaction between a therapy and a person in motion across years: these are not bounded problems with dense priors. They are sparse, partly observed, and partly unknowable. Models trained on what we have so far hit a ceiling that comes from us, not from them. And the emergent properties of AI can only stretch so far from the training data into new territories of scientific discovery, thus not a reliable hedge.

The strategic consequence of this asymmetry is that value shifts toward the inputs the model cannot supply itself. The argument that follows is an argument about location, and not about pace: AI is fast but the constraints move faster and can be fatal.

II. Discovery in 2031: abundance without advantage

By 2031, candidate molecules, targets, and mechanistic hypotheses will be generated faster than we can plausibly test them. Discovery platforms will be cheaper, broadly available, and increasingly indistinguishable across organizations. The position that depends on owning a discovery engine ages poorly, because abundance usually confers no advantage. You cannot win at a thing everyone has.

This is the first reordering, and it is worth naming clearly, because it inverts a decade of strategic instinct. The companies that bet their next decade on proprietary discovery platforms are betting on a scarcity they themselves will help dissolve. The molecules that emerge will be high-volume, sometimes elegant, occasionally truly novel, and often undifferentiated in origin. The question that will matter in 2031, more than which model produced a candidate, is which organization can test that candidate most informatively in actual patients, on a timeline a competitor cannot match.

III. Clinical development in 2031: where the bottleneck moves

Once hypotheses are cheaper and more ubiquitous, the binding constraint becomes validation. This is the move worth dwelling on, because it is not how the press currently frames the field. AI-discovered molecules in the clinic so far have cleared early-phase safety testing at unremarkable rates and have not been more successful in efficacy trials. Many still fail in humans because the underlying biology of the disease remains poorly defined. The model can write a faster answer to a question we are still asking poorly.

The trial of 2031 may start to look different from the trial of 2026 in ways that compound. AI-enabled patient selection. External and synthetic control arms where regulators permit. Biomarker-rich adaptive designs. Digital endpoints, where regulators permit. Real-world data integrated rather than appended. The organizations that run those trials best, smaller, faster, and more informatively, will define the next decade in the way the organizations that ran the first kinase and checkpoint inhibitor trials defined a previous one.

The point worth naming, because it runs counter to the prevailing framing, is that clinical development becomes more central, not less, as AI matures upstream. The bottleneck does not disappear when discovery becomes abundant. It relocates. Value follows it. And the bottleneck does not arrive in clinical development alone. It arrives, more deeply, in the biology the clinical trials are testing.

IV. Disease biology: the constraint that cannot be owned

Adjacent to the clinical bottleneck is a problem that it makes visible. When AI-discovered molecules fail in humans, the failure is usually not in the molecule, and usually not in the trial. It is that the biology of the disease was incompletely understood when the molecule was designed. The engineering layer of AI in drug discovery, the protein structures and molecular designs that the field has begun to produce at scale, sits above this absent layer. It cannot replace it. The model could not have known what was not in its training data, and the field had not yet captured the biology in a form the model could read.

This is the upstream constraint and it is the one most easily mistaken for a problem AI will solve on its own. By 2031, the binding question across many therapeutic areas will not be “can we generate a candidate” but “do we understand this disease well enough for any candidate to work.” Multimodal data on actual patients, genomics linked to transcriptomics linked to proteomics linked to imaging linked to longitudinal clinical experience linked to the patient’s own account of what changed, is the substrate from which understanding accumulates. It is also, again, the input AI cannot manufacture on its own.

I must point out that a caveat lives inside my argument: even when disease biology is finally understood, the insight may not confer a durable competitive advantage. Fundamental biological breakthroughs are hard to own. The PD-1/PD-L1 checkpoint axis emerged from shared academic work in Kyoto, Houston, and Boston, with the canonical checkpoint inhibitors distributed across multiple sponsors in close sequence (albeit initially concentrated in a single company that happened to sit on the right therapy at the right time following a multiasset acquisition). CRISPR-Cas9 mechanism diffused into thousands of laboratories within a year of the foundational paper, with downstream therapies developed by several entrants in parallel and the underlying biology eventually litigated rather than monopolized. GLP-1 receptor biology was public for decades before the molecules that crystallized it into a commercial category were developed. The foundational breakthrough in protein structure prediction, the most consequential computational result of the last decade, was open-sourced into something close to public infrastructure within a year of its release.

The pattern is consistent. Breakthroughs in biology move the field forward, not single firms within it. The organizations that capture advantage are typically the first to translate the insight into a specific molecule, a specific trial design, or a specific commercial channel, and the advantage they capture is often in the implementation, not in the underlying biology. Speed and luck matter more than understanding, once the understanding exists. This is a bit uncomfortable to write, because it implies that the most ambitious scientific work, the work whose value to patients is largest in absolute terms, is also the work whose value to a particular company is the most contingent on timing, and sometimes luck.

The strategic reading here is the same as the rest of this essay, with one addition:

invest in the biology, because the biology is the binding constraint on whether anyone’s molecule works

I think it is also critical not to assume that investment alone will yield a proprietary position. The position, when it comes, may be at the next layer down: which organization could assemble the multimodal data to see the insight first, which organization could expeditiously design the study and conduct the trial that converts the insight into proof, and which organization already holds the channel into the patient population the insight reorders. By 2036, the insight itself may belong to everyone, at an accelerated pace that will surpass today’s publication cycles in peer-reviewed journals and abstract embargoes at scientific meetings. The implementation, if you were fast or lucky, will be yours for a window.

V. Human data in 2036: what cannot be cloned

If the five-year story is about relocation, the ten-year story is about fidelity.

By 2036, the models will have converged. The performance gap between the best and the second-best discovery model, the best and the second-best regulatory drafter, and the best and the second-best biomarker pattern recognizer will narrow toward a band in which the model itself is no longer the differentiator. What will differ is what each was trained on and what each can continue to be trained on.

Here, I think, is the scarce input of the next decade: high-fidelity human biology. Longitudinal, multimodal, reuse-consented data on actual patients across actual disease courses. Imaging linked to molecular data linked to outcomes linked to patient experience, persisted long enough to support questions that cannot be answered inside a single traditional clinical trial. An organization that has assembled this kind of position cannot be cloned by a competitor with the same access to off-the-shelf models, because the inputs cannot be re-derived from public data or purchased after the fact.

Treating data as residue, rather than as infrastructure, compounds across a decade the same way poor measurement compounds across a single trial. The errors propagate. Models trained on the result magnify them. The magnification reads, to readers and to regulators, as confidence. Models trained on proxies for biology learn the proxies. Models trained on real biology, captured deliberately, learn the biology. The choice between residue and infrastructure has always been available. The window in which making it correctly still confers a durable competitive position closes inside this ten-year frame.

VI. Commercial interaction in 2036: the agent at the door

The commercial and medical interaction layer of the industry, the way a therapy reaches a prescriber and a patient, is being rebuilt around AI agents on a horizon shorter than ten years. Field force economics, medical information delivery, payer engagement, patient support, formulary access: each of these can move toward AI-mediated interaction at scale. A trusted channel, once established, is itself scarce.

The risk inside this transition is the same as the risk in every previous transition. Trust mediated through a new layer is easier to lose than to build. Patients and clinicians who feel routed rather than supported, who feel informed by something that is selling rather than serving, withdraw. The first generation of mediated channels will be remembered by the people who lived through them, in the same terms as the first generation of patient portals: as either the thing that worked, or the thing that proved the field had not yet earned the right to mediate.

VII. Coda: capex is not capability

The capability that defines biopharma in 2036 will not be ownership of the best model, because the models will converge, and because the model itself is an abundant complement to a scarce input. The capability will be ownership of the highest-fidelity view of human biology that any organization has assembled, and the proven ability to turn a prediction made by any reasonable model into regulatory-grade robust proof, into a patient outcome, and into a trust relationship, faster than competitors with the same model access can do the same work.

The tactical move, available to every competitor and conferring no advantage, is to buy AI in order to do today’s work faster. The strategic move, available now and not later, is to take a position in what AI will still depend on in 2036 and cannot supply for itself: deliberately captured human data, evidence generation as a core competence rather than a vendor relationship, and trust as infrastructure rather than as a campaign.

The reason to write this down in 2026, and not at the end of the decade, is that the most consequential decisions on each of these are decisions about contracts, about consent frameworks, about institutional posture, and about which functions are held close and which are released. These decisions have long lead times. The window in which they are still cheap is narrow.

My closing observation belongs to a longer argument: AI is not what finally lets biomedicine outrun its measurement infrastructure. It is the thing that finally makes the measurement infrastructure load-bearing in a way it was never designed to be before. A poor instrument used by hand produces a poor reading once. A poorly instrumented model produces a poor reading at industrial volume, every day, with rising confidence. My preconditions in the preamble are either what the next decade is built on or what many of our hopes and dreams crack against.

If we play our cards right, AI does not save the field. The field saves itself, using AI as one of several instruments, and emerges in 2036 with measurement scaffolding worth the name and a generation of treatments developed on it. If we do not, AI is the loudest decade in a long sequence of decades in which the field promised more than its instruments could verify, and the consequence of that, this time, reaches further than the consequence has reached before.

The shape of 2036 is being written in 2026, in the contracts being signed, the consents being framed, the endpoints being defended in regulatory meetings, and the willingness or unwillingness inside organizations to keep saying out loud that capex is not capability.

The writing is a slow choice and a quiet one. And I believe it is also the only one.

Further reading

Subscribe now

I. The ghost on the deck

There is a story I have told, in one form or another, for a long time. It is a story about a ship.

In 1747, aboard HMS Salisbury, a young naval surgeon named James Lind took twelve sailors whose gums had blackened and whose limbs had begun to fail them, and he divided them into pairs. To one pair he gave cider. To another, dilute sulfuric acid. To others, vinegar, seawater, a purgative paste of tamarind and cream of tartar. To the last pair he gave two oranges and a lemon a day. Within six days, the citrus pair was nearly recovered. The others were not. Lind had, almost without meaning to, invented the controlled trial. He had also, in the same gesture, conjured a ghost that would haunt medicine for the next three centuries: the Average Patient. A statistical phantom that exists nowhere except in spreadsheets, and yet around whom we built every regulatory fortress, every approval pathway, every billion-dollar trial that followed.

I told this story once, years ago, to a young contractor at the FDA at a meeting about something else entirely. His name was Jeremy Walsh. He listened the way he always does, carefully and without performance, and when I had finished, he asked the question I have come to recognize as his particular gift. He asked it again today on a stage at White Oak, MD, as the FDA’s first Chief AI Officer, in front of a room full of cameras, reporters, and pharmaceutical executives:

Wait. We are still doing it that way?!

In our meeting years back, I told him that yes, more or less, we were. That what changed between the Salisbury and the modern clinical trial ecosystem was not the basic shape of the enterprise but the scale at which we did it and the bureaucratic stratigraphy we had laid down on top of it. Sponsors run trials. Sites collect data. The data is packaged, cleaned, narrativized, and shipped across in pages of PDF, thousands of pages in a typical IND, some of it not even human-readable. Counts of counts of counts. Tables of tables of tables. The regulator sees the data only after the sponsor has finished telling itself the story of what the data means. Lind would have recognized the structure. He would not have recognized the latency. In a way, we have actually regressed.

My conversation with Jermey became a white paper. The white paper went up the chain. Nothing happened, which is what usually happens, and Jeremy went back to his other work. But he kept asking the question. He asked it for years. He asked it as a contractor and he asked it after he became Chief AI Officer at the FDA and he asked it in every meeting where someone told him the way things were done was the way they had to be done. Today he got to answer it.

II. What was announced

Today, the FDA announced the real-time clinical trial initiative involving AstraZeneca’s TRAVERSE study in mantle cell lymphoma, with sites at MD Anderson and Penn and Amgen’s STREAM-SCLC trial in small cell lung cancer. A Request for Information also opened today. A broader pilot program launches this summer. Commissioner Marty Makary, who led the announcement, framed the rationale precisely. For sixty years, he said, we have been conducting clinical trials in essentially the same way, and the lag between data and decision has not been a feature of safety. It has been sediment. The accumulated weight of a system that was built when data lived in filing cabinets and submissions arrived by truck. The agency, he said, can reduce that lag without cutting any corners on safety, which remains the FDA’s number one priority. He said the line plainly, and he said it more than once.

This is the part of the announcement that most deserves attention, because it is the part that is easiest to misread. What the FDA is doing is not deregulation. It is a re-engineering. The protocols are still protocols. The investigators still consent patients. The reviewers still review. Paul Burton, Chief Medical Officer at Amgen, was deliberate on this point. Pragmatic real-time clinical trials, he said, do not mean clinical research with less rigor. The fundamentals of good science do not change. Protocol development, protocol oversight, informed consent, monitoring, strong governance, FDA review. All of it remains absolutely standard and absolutely essential. Patient safety and data integrity, he said, will always be the north star.

That language matters because the architecture being introduced is one in which safety is not preserved by latency but expressed through signal. Today’s announcement is, in this sense, a strengthening of efficacy-signal detection and pharmacovigilance, not a relaxation of them. Makary made the point directly in the press Q&A. Safety signals and clinical endpoints are agreed upon by the sponsor and the FDA in advance, as they have always been. What changes is that those signals can now be transmitted to the agency in real time rather than packaged into a submission months or years later. During the pilot, sponsors will continue to submit data through the conventional pathway in parallel, so that the agency can compare the two channels and refine the model. Nothing has been removed from the safety apparatus. Something has been added.

What changes, then, is the latency. What changes is the assumption, baked so deeply into the architecture of clinical development that we forgot it was an assumption, that the regulator should see the data only after the sponsor has packaged it. Jeremy described the new model on stage in language that I suspect will sound radical to people outside the field and obvious to people inside it. We do not need everything we have ever seen. We need signals. We need to define, with the sponsor and the FDA together, what would actually change a regulatory decision, and we need to see those things happen as they happen. Not a year later. Not at the next data lock. Now.

III. Listening to the patient instead of to the form

The first time I worked with Jeremy was at the National Cancer Institute, where we designed a clinical trial in adult patients with cancer using voice recognition and facial recognition to estimate pain and mood. The patient sat down. The cameras and microphones did the rest. No survey. No clinician translating an expression into a number on a Likert scale. The signal came directly from the human being in the room, in real time, captured continuously, with a fidelity that no questionnaire has ever achieved or could. That dataset, to my knowledge, remains the largest of its kind.

The deeper observation that emerged from that work, and that I think is the actual intellectual core of what was announced today, is that the data we collect in clinical trials is not always a faithful image of the patient. It is a heavily compressed, heavily edited, heavily delayed reconstruction. The patient experiences the disease in continuous time. We sample it in discrete time. The patient lives in signal. We submit in summary. Every step in the pipeline from bedside to FDA reviewer is a lossy compression, and the loss is not random. It systematically privileges what is easy to measure and structure over what is meaningful and continuous. The Likert scale erases the wince. The case report form erases the silence before the patient answers. The quarterly data lock erases the morning the toxicity actually appeared.

What changes when signals flow in real time is not just the speed of regulatory decisions, although that matters and it matters enormously to the patient who is in hospice waiting for the drug. Jennifer Litton, the Chief Clinical Research Officer at MD Anderson and one of the academic leaders behind the AstraZeneca trial, made the point with the kind of moral clarity that only someone who has stood at the bedside can make. She is an oncologist. She is the daughter of two parents with cancer. She was diagnosed with cancer this year, and finished surgery and radiation. When you do not know whether the next patient who needs a clinical trial is going to be you or someone you love, she said, you want that trial available right now. The latency in our current system is not abstract. It is measured in funerals.

What changes, then, is also the resolution of what the regulator can see. We move from low-frame-rate snapshots of an aggregate to a continuous record of individuals. The Average Patient, the ghost we summoned on the deck of the Salisbury because we had no other way to reason about populations at scale, becomes optional. Not abolished. Optional. We can still aggregate when aggregation is what the question requires. But we are no longer forced to aggregate as a precondition of being able to see anything at all.

IV. The renovation

There is a temptation, when an announcement of this kind lands, to declare that the old order has fallen and a new one has taken its place. That is not what has happened today. Two trials. One technology partner (Paradigm Health). This is the first sentence of a new chapter, not the chapter itself. The hard work, the transformation, is the scale-up. It is the second sponsor, and the tenth, and the hundredth. It is the standards work that has to happen quietly behind the announcements, the EHR-to-EDC plumbing, the sponsor data systems that have to be re-architected, the reviewer workflows that have to be redesigned around streams instead of submissions. It is, as one industry veteran told me afterward, the long stretch where we will have to run both systems in parallel, the old way and the new way at once, before the old way can finally be retired.

There is also the human dimension, which is easy to miss in the technical conversation. Emma Meagher, who runs clinical and translational research at Penn and is one of the principal investigators on the AstraZeneca trial, made an interesting observation. She has been a trialist on the front lines for thirty-two years. She knows, better than almost anyone, how punishing the current process is on the people who actually run it. The research coordinators. The trial nurses. The project managers. They are virtually impossible to find and harder to keep. What this innovation does, she said, is not just shorten the FDA’s timelines. It eases the academic interface of patient care. It gives the workforce that sustains clinical research a reason to stay. The renovation is not only of the regulatory architecture. It is of the human one.

What Makary said almost in passing was, I think, the line that we should remember. He told the room, with a self-deprecating shrug, that he was not going to show a PowerPoint, because PowerPoint had traumatized him over the course of his career. He showed one graphic. The drug development and approval process, beginning to end. Each phase a box. Each box, he said, currently being re-engineered. The graphic is the same graphic we have been showing for sixty years. The boxes are the same boxes. What changes now is what flows between them, and how fast, and at what fidelity, and to whom.

This is not a revolution. Revolutions tear down. This is something rarer and harder. This is a renovation of a working building, conducted while the building remains occupied by patients who are, right now, today, waiting for the next box to open. The ghost of the Salisbury, the Average Patient who never existed, can finally begin to be retired. Not because we have stopped caring about populations, but because we have learned, at long last, how to listen to individuals in real time, and to aggregate that listening without losing it.

To Jeremy, congratulations. He kept asking the question for years after most people would have stopped. To the team at FDA, the sponsors, the academic centers, the technology partners who made this real, congratulations. To everyone watching from outside the agency wondering whether this is a small thing or a large thing, I will say this. It is an important thing. And in ten years it can hopefully look like the thing that broke the dam.

The signal is moving. The ship has finally left the harbor. Now we have to learn to navigate.

Subscribe now

Every oncology drug approved based on tumor-based endpoints in the last twenty-five years rests on an instrument. The instrument is not a machine, not a molecule, not a laboratory assay. It is a rather arbitrary convention. A rule about how to press a ruler against a shadow on a screen and decide whether the shadow has grown. The convention is called RECIST (Response Evaluation Criteria in Solid Tumors) and it dictates, with the stubborn authority of orthodoxy, how major regulatory decisions in solid tumor oncology have been made since the late 1990s.

RECIST works like this: A radiologist examines a CT scan and selects up to five “target lesions,” the largest measurable tumors in a given organ. She measures each one along its longest diameter, sums those diameters, and compares the sum to a baseline. If the sum has shrunk by 30% or more, she writes partial response. If it has grown by 20% or more, she writes progressive disease. Anything in between is stable disease. Complete disappearance is a complete response. These four words, expanded into the binary of ORR (objective response rate: the fraction of patients who achieve complete or partial response), are what we submit to the FDA as evidence of efficacy. They are the basis on which drugs are approved, priced, and prescribed to patients with cancer.

The framework has the unusual property of being both ubiquitous and invisible. Few oncologists have heard of it. Almost no one in clinical practice actually uses it. It is the instrument by which drugs are certified, not the instrument by which they are administered. This is the first fact about RECIST worth holding onto. We will return to it.

I. Two readers, one scan

In 2017, while at the FDA, my colleagues and I pooled the paired radiology assessments from a decade of solid tumor registration trials. The structure of these trials has a quirk useful to our purpose: for regulatory submission, every scan is read independently by two qualified radiologists, one primary investigator review and the other a blinded radiologist at a central reading facility, each blinded to the other’s work. If they disagree, an adjudicator breaks the tie. The raw data, kept for internal auditing, records how often two expert readers, looking at identical images and applying identical criteria, reach identical conclusions.

Our pooled sample contained 13,677 paired assessments. The raw concordance rate was 69.5 percent. Two of every three paired reads agreed. The remaining third did not. In hard-to-measure malignancies such as pancreatic and ovarian cancers, the discordance exceeded 40% and, in some cases, reached the predictability of a coin flip.

This figure is higher than it ought to be. In the language of measurement theory, it corresponds to a weighted kappa somewhere between 0.30 and 0.55. Kappa is a statistic that asks how much two raters agree beyond what you would expect from chance alone; a value of 1.0 is perfect agreement, 0.0 is no better than coin flips. The number we observed sits at the lower boundary of what Landis and Koch, in their widely cited 1977 classification, called “moderate.” Moderate is what you would expect from a diagnostic tool in early development. It is not what you would expect from the instrument on which the entire oncology drug development enterprise rests.

The standard defense of the framework, when this figure is raised, is procedural. Discordance, the argument goes, is non-differential. It is distributed symmetrically across treatment and control arms. Therefore it does not invalidate the comparison. The p-value remains honest. This is partly true for randomized studies. It is also profoundly incomplete.

II. The attenuation

Here is what the procedural defense leaves out, beyond the fact that many cancer drugs are advanced in single-arm studies. When you inject random error into a comparison of two proportions, the observed difference between the proportions does not stay the same. It shrinks. This is a known result in epidemiology, tracing at least to the work of Irwin Bross in the 1950s, but it is rarely invoked in discussions of clinical trial design because most people prefer to believe that their measurements are not noisy.

The mathematics is merciful in its clarity. If the true response rate in the treatment arm is P(T) and the true response rate in the control arm is P(C), and each patient is misclassified with probability m, the observed difference between arms is the true difference multiplied by the factor (1 - 2m). At our measured discordance rate of thirty percent, that factor is 0.40.

A drug whose true advantage over standard of care is twenty percentage points of ORR, a substantial effect by any reasonable standard, appears in the trial record as a drug with an eight-point advantage. The biological signal has been attenuated by sixty percent before the statistician ever sees it.

Attenuation increases the sample size. Because the number of patients required to detect a given effect scales as the inverse square of that effect, a trial that would need two hundred patients to detect the true difference would need, under this measurement regime, approximately one thousand two hundred and fifty to detect the attenuated one. Most trials do not enroll 1,250 patients. Most trials are powered to detect the signal they were designed to detect, assuming the measurement faithfully transmits it.

The directional consequence is the one that matters, and it is rarely stated plainly. Random measurement error biases observed effects toward the null hypothesis. At the level of the measurement substrate, this produces a systematic excess of false negatives over false positives. A trial that fails to reach statistical significance is not necessarily a trial of an inactive drug. Some meaningful fraction of negative trials in solid tumor oncology are trials of active drugs evaluated with an instrument that could not see them.

No one knows the size of that fraction. The instrument, by its nature, erases the evidence of what it failed to detect.

III. Four boxes for a continuous world

The second property of RECIST is a property of encoding rather than of measurement. The underlying biological variable, the percent change in tumor burden over time, is continuous. It takes any value on the real line, bounded below by minus one hundred percent and above by whatever the tumor can manage. The framework transforms this continuous variable into an ordinal category with four levels, and then, for ORR, collapses those four levels into a binary. Something biologically rich becomes something statistically thin.

There is a way to quantify the loss, borrowed from information theory. A continuous percent-change measurement carries, under plausible distributional assumptions, between 5 and 8 bits of information per patient per scan. (A bit is the unit Claude Shannon introduced in 1948 to measure the content of a signal: one bit is the information gained from the outcome of a fair coin flip.) A four-level ordinal carries two bits, which is the base-2 logarithm of 4. A binary outcome carries one. The RECIST encoding, at its most generous, transmits at most a quarter of the information contained in the measurement it encodes. At ORR, it transmits less.

The effect is not merely a loss of resolution but a loss of biological distinguishability. An agent that produces a uniform 25% shrinkage in every treated patient, a remarkable result implying a real mechanism of action operating reliably across a population, can yield an ORR of zero. It is categorized, on the endpoint, as indistinguishable from placebo/no treatment. It is also indistinguishable, at the endpoint, from an agent that produces 19% growth. Three biologically distinct populations, trivially separable in the underlying distribution of percent change, collapse to the same point in the reported outcome.

This is the structural consequence of mapping a continuous variable through a step function. The step does what steps do. It discards what lies between its treads.

IV. Stochastic at the edge

The thresholds themselves introduce a third pathology, one that becomes vivid if you imagine the patient whose true response is not in the middle of a category but at its boundary.

For example, let’s look at a patient whose tumor burden has decreased by 28%. Two percentage points below the 30% partial response threshold. Stable disease by the rules, but barely. Suppose the inter-reader standard deviation on the sum of diameters, consistent with the discordance data, is about ten percentage points. The probability that a given reader assigns this patient to the PR category, under a normal approximation, is the probability that the reader’s estimate falls below -30%, which works out to about 42%.

Forty-two percent. The same biology, the same scan, the same criteria, yields partial response in forty-two of one hundred reads and stable disease in fifty-eight. The categorical outcome is, at the boundary, approximately a coin flip. And the reported trial effect depends on how many of these coin flips land on the side that counts.

This threshold instability does more than add variance. It creates a selection pressure that propagates backward into the development pipeline. Because ORR binarizes sharply at minus thirty percent, a sponsor who wishes to maximize the probability of clearing that threshold is rewarded for advancing agents capable of producing deep responses in a subset of patients. Agents that produce shallow but consistent cytoreduction across a population, even when the underlying biology is meaningful, are structurally disfavored. They cannot clear the threshold often enough to generate an ORR that the machinery can see. And our machinery is impatient. When meaningful tumor responses are not seen in early clinical trials, the entire program is at risk.

The shape of the modern oncology pipeline, its preferential enrichment for drugs that drive responses visible to the human eye, is a direct consequence of our endpoints. We have been selecting not for what can prolong survival or increase quality of life, but for what our instruments can detect.

V. The instrument that does not exist outside the trial

The third property is the most easily stated and the most commonly overlooked. RECIST is a rather artificial clinical trial instrument. It is not a clinical tool. Community oncologists, the people who actually treat the overwhelming majority of patients with cancer, do not compute sums of diameters. They do not segment target lesions into protocol-defined categories. They read narrative radiology reports written in ordinary English (”stable appearance of the right upper lobe mass”) and make decisions on the basis of clinical judgment integrated with that narrative.

Which is to say: the endpoint that certifies a drug’s efficacy in the trial setting is not the endpoint that governs its use in the practice setting. The variable on which approval rests is not present in the data that routine care generates. This is the oncology equivalent of approving, say, statins on the basis of LDL reduction only to find out that community physicians use blood pressure to administer and titrate therapy.

When we attempt to validate oncology clinical trial findings against real-world evidence, using electronic health records and community radiology reports, we cannot reconstruct a RECIST assessment. The comparison has to be made at a different level of abstraction, using different variables, and the correspondence between the two settings, at the level of the individual patient or the individual drug, remains structurally unverifiable.

The conventional response to this observation is that RECIST is a surrogate for clinical benefit rather than the benefit itself. A surrogate is a measurable variable used as a proxy for the outcome of real interest, which in oncology is almost always overall survival. The real question, on this view, is whether RECIST-based endpoints predict overall survival. The meta-analytic answer, accumulated across hundreds of trials and now reasonably mature, is that they do so imperfectly. PFS-to-OS correlations in solid tumor oncology routinely yield coefficients of determination below 0.5. ORR-to-OS correlations are lower. The surrogate explains less than half of the variance in the outcome it is supposed to predict.

The framework persists, then, less because of its predictive validity than because of its institutional entrenchment. It is the thing we all agreed to use. Agreement has its own gravitational pull, and for a long time the conditions that would permit disagreement did not exist.

VI. What the endpoint could not see

An instrument with 30% reader discordance, indicating moderate agreement by standard statistical classification, can attenuate observed effects by 60% and inflate required sample sizes sixfold. The attenuated signal is then projected onto an ordinal encoding that transmits at most two bits per scan and produces stochastic categorization at the thresholds where the reported effect is determined. The resulting endpoint is computable only within the trial setting and cannot be reconstructed from the data generated by clinical practice after approval.

The directional bias of the whole apparatus is toward false negatives. It favors drugs whose mechanism produces a small number of responses that are deep enough to be detected by the human eye over drugs whose mechanism produces a large number of modest responses, but can prolong survival or improve quality of life. It cannot see agents whose signals fall below the threshold for categorical instability, and it cannot distinguish, at the endpoint, between agents that differ by biologically meaningful amounts.

The statistical machinery layered on top of this substrate, all the careful p-value thresholds and hierarchical testing procedures and alpha-spending rules, is bounded by what the substrate permits. No refinement of the hypothesis test can recover information that the measurement did not transmit.

The cost is not just borne by the sponsors whose trials failed. It is borne by the patients whose therapies were never developed because the endpoint could not see them.

VII. The substrate is replaceable now

The conditions that produced RECIST in the late 1990s are no longer operative. Volumetric segmentation of tumors, computationally prohibitive a few years ago, is now a solved problem. Foundation models for medical imaging, trained on orders of magnitude more data than any individual radiologist will read in a career, are mature. Automated assessment at scale, across both trial and community settings, is a deployed capability.

Over the past several years, I have been working on a replacement substrate at Project Data Sphere. We call it the Total Tumor Burden Index. It is volumetric rather than diametric, which reduces reader-dependent variance. It is continuous rather than categorical, which eliminates both the information loss of ordinal encoding and the instability at categorical boundaries. It is machine-generated rather than reader-dependent, which removes the dominant contemporary source of measurement noise and allows deterministic reproducibility across sites and across re-reads. And it is equally deployable in trial and community settings, which means the trial measurement and the practice measurement can, for the first time, be the same variable.

The regulatory path will take time. Prospective comparison with RECIST in contemporary datasets will take time. Integration across imaging infrastructure will take time. None of these barriers is technical. The technical conditions for the replacement are being satisfied with increasing precision.

VIII. Coda

The measurement substrate in solid-tumor drug development is not neutral. It has been shaping the pipeline for twenty-five years in ways the field has not made fully explicit to itself. The attention has remained on the superstructure, the p-values, the statistical analysis plans, and the regulatory pathways, when the leverage is actually in the layer underneath. Replacing the measurement, rather than refining the statistics computed upon it, is the tractable path forward. The conditions for that replacement are, at last, in place.

What we measure determines what we can find. What we can find determines what we develop. What we develop determines who is treated. The ruler we have been pressing against the tumor for a generation is not the only ruler available. It is simply the one we inherited. And that inheritance is due for review.

Share

There is a principle in pharmacology that took medicine decades to fully absorb. Liposomal doxorubicin and free doxorubicin are the same molecule. Yet one accumulates preferentially in tumors while the other damages the heart. The liposome does not make the drug more potent. It changes where the drug goes, what it encounters, what it spares; by exploiting the leaky vasculature of the tumor microenvironment and staying encapsulated in the tighter junctions of cardiac tissue. The container shapes what the active compound can do.

The same principle, it turns out, applies to AI systems. And a paper published at the end of March 2026 by researchers at Stanford, MIT, and KRAFTON gives us the clearest demonstration yet of what that means in practice and what it makes possible.

What the paper found

The paper is called Meta-Harness: End-to-End Optimization of Model Harnesses. Its subject is the code that wraps a large language model, the layer that determines what information the model sees, in what order, retrieved from where, and framed how. The authors call this the harness. It encompasses three things: memory, meaning what the system retains between interactions; retrieval, meaning what it fetches from external sources at each decision point; and prompting, meaning how context is assembled and sequenced before the model reasons about it.

The central observation is straightforward. Research cited by the authors has shown that changing only the harness around a frozen, untouched model can shift performance by a factor of six on the same benchmark. Same weights. Different wrapper. Until now, harnesses have been built entirely by hand: engineers designing prompting frameworks, retrieval strategies, and memory structures through iteration and intuition, exploring a small fraction of what is possible.

Meta-Harness automates this process. It works through a recursive loop. A candidate harness is proposed, evaluated against a set of tasks, and everything is saved in raw, uncompressed form, not a score, not a summary, but the full transcript of every prompt the model received, every response it gave, every tool call it made. A coding agent then navigates that filesystem, reads what it needs, and proposes an improved harness. The loop repeats roughly sixty times in a full run.

What distinguishes this from every prior approach to automated optimization is the refusal to compress. Systems like OPRO, TextGrad, and AlphaEvolve hand the optimizer a digest: a scalar score, a short critique, a recent snippet. Meta-Harness gives the agent the raw logs and trusts it to navigate selectively, reading a median of 82 files per iteration across the full history of prior attempts. That access to raw diagnostic history, rather than compressed summaries, turns out to be the key design decision.

The results across three domains are real and substantial. On a text classification task where a model updates its memory from labeled examples and is evaluated on held-out cases, the discovered harness outperforms the best hand-designed system by 7.7 accuracy points while using four times fewer tokens. On 200 olympiad-level mathematics problems, a single harness discovered during search improves accuracy by an average of 4.7 points across five AI models never seen during the search process, indicating the discovered strategy is genuinely general. On TerminalBench-2, a benchmark of 89 real-world tasks requiring AI agents to autonomously compile code, debug systems, and complete multi-step terminal workflows with full automated verification, the discovered harness places first among all agents running on the same underlying model, surpassing every hand-engineered competitor.

But what the paper demonstrates is not just that automated harness search works. It is that the agent doing the searching behaves in ways that matter for how we think about these systems going forward.

The agent that reads its own case files

The paper’s appendix traces, iteration by iteration, how the agent reasoned through a live search run on TerminalBench-2, starting from a strong hand-engineered baseline. What emerges is not a description of brute-force search. It is a description of systematic diagnosis.

In the first two iterations, the agent bundled a structural bugfix with a rewritten prompt template. Both regressed from the baseline. In the third iteration, rather than proposing a third fix, the agent looked back across both failures. It noticed that while the two attempts differed in their structural changes, they shared one intervention: the prompt rewrite. It separated the confounded variables and wrote verbatim into its log: “the structural bugfixes were confounded with harmful prompt changes.”

This is abductive reasoning, the same logic a careful physician applies when two patients on similar regimens both deteriorate. You do not conclude the drug failed. You ask what they had in common. You isolate. You test. The agent was performing differential diagnosis on its own previous attempts, using the filesystem as its case history and the execution traces as its clinical notes. It could only do this because it had access to the raw logs. A compressed summary would have hidden the shared intervention entirely.

By the seventh iteration, after six consecutive regressions that taught the agent where the system’s fragility lived, it stopped modifying the existing machinery and instead added something before the loop began: a snapshot of the computational environment injected into the initial prompt. Its recorded reasoning: avoid touching the previously fragile components. It had learned not just what failed but why and used that understanding to reason about risk.

This behavior (reading raw history, identifying confounds, forming causal hypotheses, reasoning about where fragility lives) is what full access to prior execution traces enables. Compressed feedback cannot support it. And it is precisely this kind of reasoning that the hand-engineering of clinical AI harnesses has always depended on human experts to provide.

The compression problem, demonstrated

To understand why this matters for biomedicine specifically, it helps to look at what happens when LLM harnesses are hand-engineered in a clinical setting.

In a study we conducted last year at Project Data Sphere with Thomas Jefferson University, four independent investigators used LLMs (ChatGPT-4 and Gemini Advanced) to analyze data from a clinical trial evaluating LY2510924 combined with carboplatin and etoposide for extensive-stage small cell lung cancer. All four investigators worked from the same dataset, the same chain-of-thought prompting framework, and the same analytical objectives.

The results were interesting but inconsistent. One investigator found a statistically significant improvement in progression-free survival with the addition of LY2510924, contradicting the original publication and the other three investigators. Another reported an objective response rate that deviated substantially from every other analysis. Cohort sizes ranged from 89 to 180 participants from the same dataset, suggesting the LLMs characterized entirely different patient populations depending on how the prompts were constructed. The conclusion of the paper was clear: variability in prompting strategies drove variability in clinical conclusions, and highly trained subject matter experts remain essential to review and interpret LLM-generated analyses.

This is the harness problem in clinical research, demonstrated under controlled conditions. The chain-of-thought framework was the harness. It was hand-engineered. Different investigators interpreted it differently. The model was sensitive to subtle prompt variations. And the outputs, including whether a drug appeared to improve survival, changed accordingly.

Meta-Harness is a direct response to exactly this problem. Rather than asking human experts to design the optimal prompting framework and then hoping investigators apply it consistently, automated harness search would systematically explore the space of prompting and retrieval strategies, read its own failures at the case level, this analysis got PFS wrong, what was it shown, what did it retrieve, what framing led it astray, and discover configurations that produce consistent, reproducible outputs across investigators, model versions, and datasets. The subject matter expert moves from designing the wrapper to evaluating and validating what the automated search discovers. That is a meaningful and practical shift in how we deploy LLM-based clinical tools.

What the 40% tells us

A drug achieves a 40% ORR in a clinical trial. We call this a success. The drug is approved, deployed, and prescribed. But 60% of patients who receive it do not respond. They experience toxicity without benefit, akin to taking a toxic placebo.

Those 60% non-responders are not a rounding error. They are a population. And somewhere in the data we already have; genomic profiles, imaging series, early response signals, treatment histories, patterns of toxicity; there may be information that distinguishes them from the 40% who responded. The challenge is not always that this signal does not exist. The challenge is often that we do not know where in the vast space of available data to look for it, and that the LLM-based clinical tools we are deploying are wrapped in harnesses that were never designed to find it.

This is where automated harness search becomes directly relevant to patient care. An LLM-based decision support tool wrapping a clinical dataset will only surface the signal that its retrieval and context construction strategy is designed to find. If that strategy is hand-engineered, it reflects the intuitions of the engineers who built it. Automated harness search would systematically explore alternative retrieval policies: what patient-level data to pull, how to frame prior cases, what early signals to foreground; reading its own failures at the case level and discovering configurations that surface signal the hand-engineered wrapper missed.

Where this applies today

Countless LLM-based tools are being deployed across biomedicine right now, each with a hand-engineered harness, each leaving performance on the table in ways that automated search could address. A few meta-harness insipred opportunities to highlight:

Clinical trial analysis. Our study demonstrated this problem directly. LLM-based systems for analyzing trial data are sensitive to prompting strategies in ways that produce clinically meaningful variation in outputs. Automated harness search could discover prompting and retrieval configurations that produce consistent results across investigators and model versions, reducing the burden on subject matter experts to catch errors and standardizing the analytical process across institutions.

Clinical decision support. LLM-based systems are increasingly used to synthesize patient records, surface relevant literature, and support diagnostic and treatment decisions. The harness determines what gets retrieved from the electronic health record, how prior similar cases are identified and presented, how uncertainty is communicated to the clinician. These are wrapper decisions. They are currently made by hand, reflecting the intuitions of the development team. Automated search over these decisions, using case-level outcomes as the feedback signal, could discover retrieval and framing strategies that improve decision quality in ways that no individual engineer or clinician would have designed.

Drug development workflows. LLM-based tools are being used to synthesize preclinical literature, generate hypotheses about mechanism, and support the design of clinical development programs. The harness determines how prior experimental results are retrieved and contextualized, how mechanistic hypotheses are framed, what evidence is foregrounded when the model reasons about a new compound. Automated harness search could discover context construction strategies that improve the quality and consistency of these outputs, not by changing the model, but by changing what it sees.

Biomarker and multi-omics analysis. When LLM-based systems reason over complex biological data, integrating genomic profiles, imaging features, proteomic signatures, and clinical records, the harness determines what gets retrieved, how it is integrated, what gets foregrounded for each patient. The design space here is enormous. Automated search over retrieval and integration strategies, reading its own failures at the case level, could discover configurations that surface signal in high-dimensional biological data that hand-engineering would not find.

Regulatory and evidence synthesis. LLM-based tools are being used to synthesize clinical evidence for regulatory submissions, systematic reviews, and health technology assessments. The harness determines how evidence is retrieved, how studies are compared and contextualized, how uncertainty and heterogeneity are communicated. Inconsistencies in these outputs, analogous to those demonstrated in our study, have direct consequences for regulatory decisions. Automated harness optimization offers a path toward more consistent, reproducible evidence synthesis.

What changes for experts

None of this necessarily replaces clinical or scientific expertise. It relocates where that expertise is most valuable.

Today, expert knowledge goes into designing the harness, crafting the prompting framework, specifying the retrieval strategy, determining what context the model sees. This is painstaking, iterative, and bounded by individual intuition.

With automated harness search, expert knowledge moves to a different layer. Experts define the objective: what does a good clinical trial analysis look like, what does a useful decision support output look like, what constitutes a failure worth correcting. They evaluate and validate what the automated search discovers. They set the boundaries of what the system is permitted to do. They interpret the discovered harness and decide whether it reflects genuine clinical reasoning or a brittle artifact of the search process.

This, I believe, is a more appropriate deployment of expertise in the age of AI. The combinatorial space of possible harness configurations is too large for human intuition to explore systematically. Automated search is better at that. The evaluation of whether a discovered configuration is clinically sound, generalizable, and safe is a judgment that requires deep domain knowledge. Experts are better at that.

The compression principle

Running through all of this is a single epistemological claim that the Meta-Harness paper demonstrates more cleanly than any prior work: compressed feedback is lossy feedback, and lossy feedback leads to lossy learning.

In the paper’s controlled ablation, the system with access to raw execution traces achieves 50% median accuracy on a text classification task. The identical system with access only to scores and LLM-generated summaries achieves ~35%. Fifteen points separate a system reasoning from raw evidence and a system reasoning from digests of that evidence. That gap is the measured cost of compression, of replacing the signal with a summary and then trying to reason from the summary as if nothing had been lost.

In medicine, we compress constantly. Individual patient trajectories become survival curves. Heterogeneous treatment responses become pooled response rates. The richness of what happened to each person in a trial becomes a table entry and, eventually, a label (responder or non-responder) that discards the nuances of the outcome and everything that led to it. And here precision matters: our prized p-value is a downstream artifact, a summary of a summary. The compression that happens earlier, in the decision to aggregate heterogeneous individual trajectories into a single outcome distribution, can make the p-value meaningless. Once that aggregation is made, the p-value reports on what remains after the flattening.

We compress because reasoning over raw trajectories at scale has been historically intractable. That constraint is loosening today. What Meta-Harness demonstrates is that a system given access to raw case-level evidence can discover things that a system given only summaries cannot and that the difference is not marginal. In clinical AI, where the heterogeneity is biological and the stakes are patient outcomes, building systems that reason from richer, less compressed evidence is not just a technical improvement. It is a different epistemological commitment, one that we are now, for the first time, practically positioned to make.

The recursion

Meta-Harness is itself a harness. The coding agent that proposes new harnesses is wrapped in a skill document defining its role, its constraints, its objective. That document is, in principle, subject to optimization too. What happens when the outer loop optimizes not just the inner harness but the proposer’s own search procedure? The loop accelerates. The engineer becomes, progressively, a curator.

This is not a distant prospect. It is the obvious next experiment given the capabilities that exist today. And it implies a trajectory in which the performance ceiling of LLM-based systems is no longer bounded by the bandwidth of the engineers designing their wrappers but by the quality of the feedback signal and the clarity of the objective.

That means the most important design decisions going forward are not which model to use or how large to make it. They are: what does success look like at the case level, how do we measure it, and how do we build the feedback infrastructure that lets automated search find the wrapper configurations that achieve it. In biomedicine, those are questions that require clinical expertise, regulatory judgment, and scientific rigor. They are also questions that, once answered well, unlock a new generation of LLM-based tools that are more consistent, more generalizable, and more useful than anything hand-engineering alone can produce.

Like liposomal doxorubicin, the harness changes what the active compound can do. Unlike the drug, it can redesign itself. And unlike the chemist, it never stops iterating.

Subscribe now

Share

Yesterday, the US National Cancer Institute (NCI) convened a workshop on foundation models in cancer biology. What struck me most was not the technical ambition on display but the quality of the skepticism, a virtual room full of people who have stopped debating whether these models belong in oncology and started asking something considerably harder: whether the frameworks we are using to build and evaluate them are capable of representing the biology we need them to predict. That question is one I have been working through for some time, and my talk was an attempt to lay out the argument as precisely as I could.

I. The Molecule that knew everything except what mattered

I opened with RG7112, an MDM2 inhibitor evaluated in a Phase I trial for acute myeloid leukemia. In silico it was immaculate: low nanomolar affinity for its target, greater than 1000-fold selectivity over off-targets, favorable ADMET profile, Lipinski’s Rule of Five cleared without objection. Every computational checkpoint had been passed. The molecule had been assessed using structure-based methods drawing on thousands of crystallographic structures and the full weight of what computational drug discovery could bring to bear.

In patients, it produced dose-limiting thrombocytopenia and neutropenia, along with systemic homeostatic disruption that none of the models had anticipated. The compound had raised p53 levels exactly as designed. What no simulation had represented was MDM2’s role in hematopoietic homeostasis, its function in maintaining the very cellular populations whose depletion closed the therapeutic window. The biology responded across tissues and timescales the model had no architecture to reach. The simulation was not wrong. It was operating in a different biological register than the one that determined the outcome.

I used RG7112 not because it failed spectacularly but because it failed instructively. Since 1950, the cost of bringing a drug to market has doubled roughly every nine years, now exceeding $2.6 billion per approval, while the enabling technologies have advanced exponentially across the same period. Drug output per research dollar has collapsed anyway. We are applying increasingly powerful instruments to a problem whose fundamental difficulty has not yielded to them -- and that asymmetry demands an explanation beyond the usual invocations of biological complexity.

II. When the abstraction becomes the obstacle

The explanation I proposed at the workshop is architectural. Most modeling approaches in drug discovery are built on a reductionist biological view: Protein A influences Protein B, which influences Protein C. The abstraction is not false, it has produced real successes, most durably when the biology happens to be genuinely linear and the target genuinely necessary and sufficient, as with BCR-ABL in chronic myeloid leukemia or PD-1/PD-L1 signaling in tumors with pre-existing immune infiltration. But the danger is in treating the abstraction as a complete description of living systems, because biological systems are dense, adaptive networks of thousands of interacting signals, most unmeasured and many unknown. When models are built on simplifying assumptions that do not hold in vivo, their predictive validity remains limited regardless of how sophisticated the computation becomes.

This was the structural failure of the first wave of AI in drug discovery, which ran roughly from the 1980s through the 2010s. QSAR models, molecular docking, virtual screening, tools that operated within closed experimental loops, trained on proxy variables like binding affinity and validated against similar proxies. The correlations they captured were real within constrained systems. What they accumulated was not causal understanding of adaptive biological behavior but increasingly confident predictions about an abstraction that diverged from reality at the exact boundary where clinical outcomes are determined.

The current generation has a different vocabulary (generative design, foundation models, phenomics, etc) but a version of the same core limitation persists. AI today is genuinely effective at optimization: navigating the tradeoff space between molecular potency, selectivity, metabolic stability, and synthetic accessibility with a precision that accelerates medicinal chemistry substantially. But optimization and discovery are different problem classes.

Optimization assumes the underlying biological hypothesis is correct and improves performance within that framework. Discovery requires determining whether the hypothesis is valid in the first place. No amount of optimization capability resolves uncertainty in discovery, and a model that efficiently explores chemical space around a target that turns out to be dispensable in the context of a living organism is threading the needle through the wrong fabric.

The phenomics approach makes this concrete. High-content phenotypic assays measure morphological or transcriptional changes in cell-based systems and use them as proxies for therapeutic efficacy. The technical achievement is real. But cell lines on plastic lack immune infiltration, metabolic gradients, and tissue architecture, the features that determine whether a perturbation produces clinical benefit in a patient rather than a phenotypic readout in a well. A model may predict phenotypic rescue accurately and clinical outcome not at all, because the proxy and the target are operating in different biological registers.

The data I presented from a study we published in 2020 makes the same point at the level of the entire translational pipeline. Across 108 oncology drugs evaluated in mouse, rat, dog, and monkey models, we found median positive predictive values of 0.65 and negative predictive values of 0.50 for predicting human toxicity. A clean animal signal provides essentially coin-flip assurance of human safety.

Our challenge is not confined to computational modeling; the entire pipeline from in vitro systems through animal models does not reliably approximate human biology, and any foundation model trained on data generated by that pipeline inherits its limitations as structural features of the training distribution.

The mathematical frame for all of this is interpolation versus extrapolation. Machine learning systems are reliable when test conditions resemble training conditions closely enough that the learned mapping holds. Chemistry and physics generally meet this condition; governing principles are stable and well-characterized. Biology does not. Emergent behaviors, adaptive resistance, immune tolerance, and compensatory signaling across tissue types are the phenomena that determine clinical outcomes, and they exist outside the training distribution of any model built primarily on experimental proxies. This is why technical validation and clinical validation remain separated by five to seven years; which generative advances have not compressed. AlphaFold predicts protein structure with near-experimental accuracy. RFdiffusion designs proteins with no natural template. These are genuine inflection points in simulation fidelity at the molecular level. A protein that folds as predicted still has to work in a patient, and that distance has not shortened.

III. The language we use is part of the problem

Running beneath all of this is a representational problem I addressed in a Clinical and Translational Science paper, and that the workshop made more urgent. The classification systems that organize cancer biology were not designed to predict therapeutic response. They were built to describe what a pathologist sees under a microscope, and they achieved institutional authority through historical convenience rather than mechanistic precision. As I often say as an illustrative example, “non-small cell lung cancer” defines a disease by negation; it tells you where the tumor was found and what it failed to resemble histologically. It groups malignancies with distinct molecular drivers, immune microenvironments, and resistance mechanisms under a linguistic label that encodes none of that information.

When these inherited categories become training labels for foundation models, the models do not simply inherit their limitations; they amplify them. More parameters and more computing power make the model more confident in the wrong abstraction. The clinical consequence is measurable: overall response rates with doublet chemotherapy in morphologically defined non-small-cell lung cancer hover around 30%, whereas in molecularly defined, tissue-agnostic contexts they routinely reach 50-90%. The classification system produces the wrong answer the majority of the time because it was never designed for the question being asked of it.

What a mechanistically grounded alternative looks like in practice is what I discussed through TTBI, the Total Tumor Burden Index we are developing at Project Data Sphere. It is autonomous, volumetric, longitudinal, and independent of inherited tumor classification categories. It aims to create a biomarker that reflects what the disease is actually doing rather than relying on a label derived from human sensory input.

Drew Williamson presented PathChat at the workshop, a multimodal language model built on whole-slide images and pathology case reports, and the evaluation findings aligned closely with my argument. The NLP metrics the field has inherited for assessing language models, METEOR, ROUGE, BLEU, measure textual similarity to reference answers. In pathology, that is not the same as diagnostic utility. A model can reproduce the vocabulary and syntax of a pathology report while conveying nothing actionable for a clinician, and standard benchmarks lack a mechanism to detect this difference. When his group convened seven experienced pathologists to evaluate open-ended model responses, what emerged was not a performance score but a structural disagreement: different pathologists had substantially different preferences for granularity, format, and the appropriate balance between completeness and precision. The ground truth the field has been optimizing against does not constitute a coherent standard, meaning every model trained on it has been learning to satisfy a consensus that does not actually exist. It is the evaluation version of the same problem I raised about classification: a methodology that accumulated institutional authority before its limitations were understood, now quietly shaping what the next generation of models learns to produce.

IV. What getting it right actually requires

The three paths forward I proposed in my talk each involve changing what models are trained to represent rather than how efficiently they represent it.

The first is multimodal longitudinal foundation models that learn biology rather than chemistry, integrating structure, function, and time so that a model can represent the relationship among TGF-beta signaling, CD8+ T-cell exclusion, and checkpoint inhibitor resistance as a coherent biological system rather than a set of independent variables. The biological context is not optional. It is the prediction target. Kun-Hsing Yu’s work from Harvard demonstrated what becomes accessible when models are built with this orientation: predicting HER2 mutation status in diffuse large B-cell lymphoma at 0.95 AUC from routine H&E slides, without molecular training data, by recovering the genomic signal that histological images contain but that conventional pathology was never designed to extract. That same capacity for hidden signal recovery, his group also showed, applies to demographic confounding; significant performance disparities across race, sex, and age in cancer diagnosis tasks, reduced by 88% across 15 independent cohorts through a contrastive learning approach that separates outcome-relevant signal from demographically correlated noise. The engineering is real. What it is compensating for is a data generation process that encodes the access disparities of a healthcare system with uneven representation, and that process will continue to produce the same confounding in the next round of training data unless something changes upstream of the model.

The second path is a shift from target-centric to systems-level modeling, treating feedback loops and compensatory signaling as the biology to be represented rather than the complexity to be abstracted away. A molecular target is an incomplete abstraction. Intervention at any node propagates through feedback mechanisms, compensatory pathways, and cross-tissue signaling that single-target models cannot represent. Eric Schadt’s patient trajectory engine from Pathos is among the most architecturally promising attempts I've encountered to operationalize this: a large language model orchestrating frozen multimodal experts across DNA, H&E, RNA, and clinical text, with a biological knowledge graph imposing mechanistic validity constraints on the reinforcement-learning reward structure. The model is penalized not just for inaccurate predictions but for predictions that are statistically plausible and biologically incoherent. In an NSCLC case he walked through, he integrated EGFR mutation status, immune microenvironment characterization, and p53 loss into a single patient-level resistance narrative, which is how oncologists actually reason about therapeutic failure, and which current classification frameworks are structurally unable to produce. Whether this architecture generalizes from compelling case demonstrations to reliable population-level predictions is what prospective trials are for. Schadt was explicit that the clinical trial is the real test, which is the right frame, and also the one that the first wave of AI in biomedicine systematically deferred.

The third path is reorienting the field’s objective toward failure prediction. The value of AI in drug development is not only accelerating winners. It is identifying losers before they consume years of capital and patient exposure. Failure modes, toxicity, off-target cascades, and adaptive resistance are, in principle, predictable given the right training signal, but the field has organized its investment almost entirely around abundance narratives. The leverage in systematic failure prediction remains largely unclaimed. Richard Pazdur used to say during my time at the FDA that sins of omission belong to a higher circle of the Inferno than sins of commission. At current attrition rates, an early no is worth as much as a late yes.

V. What passing the test would actually look like

Among the day's presentations, Susan Galbraith's account of AstraZeneca's computational pathology work stood out for a specific reason: it had outcome data. The TROP2 biomarker she described was not a refinement of conventional IHC scoring but a replacement for it: a continuous, normalized ratio derived from subcellular membrane quantification that captures internalization rate as a direct mechanistic determinant of ADC efficacy, rather than a categorical approximation of staining intensity that has historically correlated with it. The result in TROPION-Lung01 was a hazard ratio of 0.52 in biomarker-positive patients and no detectable benefit in biomarker-negative patients. That degree of outcome stratification does not emerge from morphological proxies in such settings. It emerges from measuring what the biology is actually doing. The EGFR mutation prediction model she presented (0.905 AUC with 96% negative predictive value from histological slides) extends the same logic to the population that tissue scarcity and access barriers can currently exclude from molecular testing.

These results point toward what I framed as the Turing test for drug discovery: not whether AI can design a molecule indistinguishable from human work, but whether it can design an intervention that produces the intended effect in humans, indistinguishable from our intent. The TROP2 work is closer to passing a version of that test.

Our simulations have gotten considerably better. But the distance between what our best models predict and what happens in patients is real, documented, and not closing as fast as the technical progress would suggest it should, because the technical progress is happening at isolated discrete levels and the problem is at the systems level, at the representational level, at the level of what we choose to measure and call ground truth.

Closing that distance requires not more compute applied to inherited frameworks but the reconstruction of those frameworks from biological first principles. That reconstruction is the work ahead. And the workshop yesterday was a useful reminder of how much of it remains ahead.

Subscribe now