Working Note 07 · The Numberwall
Seven in ten drugs that start Phase 2 never pass it.
Phase 2 is the first time a new drug meets many different sick people, over months, on the outcome it will be sold on. Before that point, models predict well. Here, they stop. Call it the wall.
Sixty years of volume
More shots made it worse.
For sixty years the industry’s answer was volume. Combinatorial chemistry, high-throughput screening and genomics made it cheaper and faster to make and test candidates, by orders of magnitude. New drugs per billion dollars fell about eightyfold. Jack Scannell and colleagues called it Eroom’s law: Moore’s law spelled backwards.
Inflation-adjusted. Scannell et al., Nature Reviews Drug Discovery 2012 (white, 1950 to 2010); Ringel et al. 2020 extension (green, 2011 to 2023). Points digitized from Scannell’s 2025 slides. Dashed line: the 1950 to 2010 trend.
The industry got faster at picking the wrong drug. The decline stopped once, after 2010. The cause is not proven, but one candidate fits the rest of this story: targets began to come with evidence from people.
What AI has done so far
AI designs better drugs. It has not beaten the wall.
Phase 1 asks whether a molecule can be made, dosed and tolerated. That is chemistry and physiology that repeat from one person to the next, and AI is good at it. Phase 2 asks whether the drug works across many different sick people, where everything that differs between them starts to matter at once.
Phase 1
Can it be made, dosed and tolerated?
Phase 2
Does it work in many different sick people?
Share of programs that pass each phase. Jayatunga et al., Drug Discovery Today 2024, who call the Phase 2 rate comparable to historic industry averages. The Phase 2 sample is small, about ten programs.
AI improved the steps that repeat and left the wall where it was. A drug can be designed beautifully for the biology in a dish and still meet a population it was never shown.
Why the split? AI learns best from data written in a code: language, and the protein sequences evolution wrote. The steps before the wall are written in one. The response of many different patients to a new drug is not.
Working Note 05: Where AI Bends NatureThe bet now being placed
Now AI is putting more shots on steroids.
The money behind AI drug design has never been larger. Isomorphic Labs raised $2.1 billion in May. Xaira launched with more than $1 billion. Lila Sciences has raised $550 million to build autonomous labs. A virtual-cell program is billed at about $1.8 billion. The pipeline was already filling before any of it: about 11,000 programs in 2015, about 21,000 by the end of 2024.
If AI follows the old pattern, more candidates arrive at the same wall. Turn up AI design and watch the clinic. Then turn up the forecast instead.
AI drug design: candidates per year
Forecast of how a drug performs in people
Designed a year
100
Never tested, a year
90
Trials that succeed
10.8%
Today: about 100 candidates designed for every 10 trials the clinic can run. About 11% of what goes into trials becomes a drug; most of the rest dies at the wall. Now turn up AI design.
An illustration on the selection model the note uses (Scannell and Bosley, 2016). The clinic can start ten programs a year, a stand-in for the patients, sites and money trials need, and it takes the most promising candidates by a forecast whose correlation with success in people is 0.1 today. One random candidate in twelve would succeed. “2× better” means a correlation of 0.2. The dots are a picture of the flow, not a count.
Faster design without a better forecast clogs the pipe. The bigger bet is a better forecast of how a drug will perform in people.
Funding: company announcements, 2024 to 2026. Pipeline: L.E.K. Consulting analysis of Citeline Pharmaprojects, January 2025.
The natural question
What would AI need to forecast how a drug performs in people?
Much of the new money is going into virtual cells, autonomous labs and design engines, all steps removed from patients. The bet is that a better model trained on that data can forecast what a drug will do in people. That bet has already been tested against a wall like this one, with every advantage: in the weather.
The butterfly effect
Forecasts have a wall too.
In the 1960s Edward Lorenz started two weather forecasts from almost exactly the same place. They agreed for a while, then bore no resemblance. Tiny differences grow until they swamp the forecast. It became known as the butterfly effect, and it gives every forecast a horizon.
The cleanest test
The best AI ever built gained half a day.
Weather has known physics, global observations every few hours and decades of records. AI forecasters now beat the physics almost everywhere. They moved the horizon by about half a day.
Day the forecast stops being useful, 2020, global. WeatherBench 2; our reading of the published files.
Forecasters never relied on a better model alone. They look again every few hours, and the AI models that beat the physics learned from decades of those looks. A drug trial asks for a harder forecast: months to years ahead, across thousands of bodies, with equations nobody knows.
Try it
See the wall. Then look again.
Start with the weather, where the physics is known. Follow steps 1, 2 and 3. Then switch to a drug and watch the same thing happen.
The forecast is useful to day 10.9
Each thin line is one forecast, started a hair away from the real weather (gold). They agree for a week, then fan apart until they fill the sky. Where that happens, forecasting stops. That is the wall. Now make the starting data better.
A brighter lamp: better starting data
Look again at the real weather
The weather is a real chaotic system: the Lorenz equations, the original butterfly effect, with small random inputs added along the way. Time is scaled so today’s forecast stays useful for about 11 days, close to the best real forecasts in 2020 (WeatherBench 2: 9.1 to 11.3 days). A forecast counts as useful while its error is under 60% of the weather’s natural range.
Side by side
A brighter lamp, or another look.
A hundred times better, even perfect, starting knowledge moves the wall a little. Looking again does not let today’s model see any further. It keeps every forecast short, so the wall is never reached.
The weather
A drug (illustration)
An open question
Looking again kills failures sooner. Does it teach AI to see past the wall?
Not on its own. A look tells you where the drug is now; it does not make the next forecast longer. Weather suggests what else a model would need. We see three ingredients, and none has been proven for drugs.
1. Start from people.
Evidence about people already moves the wall. Targets backed by human genetics, and by a mouse whose knocked-out gene reproduces the disease, pass Phase 2 far more often. Even with both, one in five fails. That remainder is the wall.
Share passing Phase 2, about 4,100 programs outside cancer (Minikel et al. 2024 program data; evidence from Open Targets). Our pre-registered test. These are associations, not proof that the evidence caused the success.
2. Look again, and fail cheaply.
In 2020 the RECOVERY trial tested many COVID treatments side by side, read one outcome nobody could argue with, death at 28 days, and dropped what did not work. In about three months it found that a cheap steroid cut deaths by about a third in patients on ventilators, on a core grant of about $2.8 million. No new AI was involved. Looking again did not make anyone’s forecast longer. It made being wrong fast and cheap.
3. Write the record.
AI beat the physics in weather because it had a record to learn from: decades of the atmosphere observed every few hours and stitched into one consistent history. Drug development has no record of how many different sick people respond to a new mechanism, read again and again over time. Trials that look often, with a randomized control alongside, would write it. Today’s trials are tomorrow’s training data.
Would a model trained on that record see past the wall?
Weather says further than today, but not forever: its best AI still stops near two weeks. For drugs, nobody knows. Finding out is the experiment that, so far, draws little of the money, and it starts close to the patient.
Three questions for any AI drug program
- How far from the human outcome does your readout sit?
- What is its measured validity against the clinical record?
- How often does your program observe people and change course?
An interactive companion to Working Note 07, Brinton Bio. The weather model is a real chaotic system; the drug pictures are illustrations of the note’s argument, not measurements or predictions about any drug, program or company. Nothing here is investment, medical or professional advice.