AI and science
The end of intellectual boundaries? AI, epistemic trespassing, and the future of scientific discovery
In October 2026, an AI agent solved 49 of 50 unpublished problems from 11 specialties of theoretical physics, after its objections sent the experts back to find errors in their own answers. Philosophers call it epistemic trespassing when an expert passes judgment outside their field. As AI makes it cheap to work in a field you never trained in, the ability to judge that work becomes the scarce part of science.
- An AI agent solved 49 of 50 unpublished problems across 11 specialties of theoretical physics, after the experts found errors in their own answers.
- The authors found agents weak at choosing interesting problems, and called the missing skill taste.
- Linus Pauling, a Nobel-winning chemist, promoted vitamin C against cancer. Two Mayo Clinic trials found no benefit.
- High HDL cholesterol goes with fewer heart attacks, yet gene variants that raise HDL did not lower the risk.
- When a lab can run only so many experiments, more ideas add discoveries only if someone picks better ones.
On 17 October 2026, Jack Clark, a co-founder of Anthropic, gave the Per Jacobsson Foundation Lecture at the IMF and World Bank meetings in Bangkok. Near the end of the speech, he read out a story from the Indian physicist Suvrat Raju about a test that Raju and his colleagues had built for AI agents.
The test held 50 unpublished problems in theoretical physics. “Our initial evaluation suggested that GPT-6-Astra’s score was in the late eighties,” Raju wrote. When the team looked at the problems the agent had missed, the models, “especially Astra, insisted that they were right and that it was the expert-supplied answers that were erroneous.” The experts went back to their own work and “found some subtle errors buried deep inside their calculations.” Once they fixed them, Astra’s score reached 98%, or 49 of 50.
First grading
The graders mark each answer against the experts’ key. Astra’s first score is in the late 80s.
The pushback
On problems marked wrong, the models, especially Astra, insist that the key is wrong.
The recheck
The experts recheck their own work and find subtle errors buried deep in their calculations.
Final score
With the key corrected, Astra’s score climbs to 98%, or 49 of 50.
Clark drew a wide conclusion from the result. If AI systems can solve problems creatively enough to succeed at science, he said, “we should expect this to be a sign of their ability to generalize into many domains, including certainly AI research itself.”
I first heard the term epistemic trespassing from the geneticist Ben Neale, during a two-and-a-half-hour chat in his office at the Broad Institute of MIT and Harvard. I was on a three-day research visit to his team in the first week of August 2026. The term describes an expert who excels in one field making confident judgments in another, where that excellence does not reach. AI makes it easy to cross into a new field. Trespassing is about what happens to judgment on the far side. As AI makes it cheap to work in a field you never trained in, the ability to judge that work becomes the scarce part of science.
Fifty problems, eleven specialties
The paper behind Raju’s story is TheoPhysEval, posted on 9 October 2026 by A. Ramesh Chandra, Raju and more than 30 co-authors. Its 50 problems come from 11 areas of theoretical physics, and each has an answer that can be checked. The authors estimate that, without AI, “a typical problem in our collection could be solved by a world expert in about a day or less but would take considerably longer for experts in adjacent specialties.” Raju was blunter. “I don’t know of any individual who can solve any significant fraction of our final collection of problems without AI assistance.”
Five agents took the test twice, once with web search and once without. GPT-6-Astra, running in OpenAI’s Codex, solved 49 problems at an average cost of under US$3 each. Opus 5.5, running in Anthropic’s Claude Code, solved 46.
49 of 50 solved
Switching off search changed little. Astra and Opus each solved 46 without it, which the authors read as a sign of “significant memorization of the literature.”
The authors call their problems “turn-the-crank” research problems, “where known techniques are used to generalize an extant result.” Solving them “does not require great originality or the ability to construct new physical theories.” Some experts used AI tools on their own problems, so the authors “cannot rule out data leakage,” and each agent ran only once. Choosing a problem worth calculating was outside the test’s scope.
The authors also describe, from their own experience, where agents fell short. When a problem proves too hard, a good researcher “can then find a related but simpler and equally interesting problem to solve,” and “current agents are not adept at dynamically switching direction in this manner.” Physicists share a rough sense of what makes a problem interesting, and “agents do not appear to have the same ‘taste’ in problems.” The paper ends by suggesting that physicists, long told to “shut up and calculate,” “might need to ‘stop and think.’”
A method travels further than its judgment
In machine learning, generalization means doing well on cases unlike the ones a system learned from. Clark’s argument is that agents which carry methods across eleven physics specialties may carry them across science. Generalization comes in degrees, though, and a method can travel further than the judgment needed to use it. Cardiology has a clean example.
People with high HDL cholesterol, often called good cholesterol, have fewer heart attacks. In a 2012 study in The Lancet, Benjamin Voight, Sekar Kathiresan and colleagues estimated that each standard deviation of higher HDL went with 38% lower odds of a heart attack. The obvious conclusion, that raising HDL would prevent heart attacks, turned out to be wrong.
In 2007, a trial of the drug torcetrapib in 15,067 high-risk patients raised HDL by 72%. It was stopped early because more patients on the drug died (hazard ratio 1.58). The investigators found signs of an off-target effect and could not rule out harm from the target itself.
Voight’s team then turned to genes, using a method called Mendelian randomization. A variant in the LIPG gene gives its carriers higher HDL for life, enough to expect about 13% lower odds of a heart attack. Their odds were the same as those of non-carriers. Run on LDL cholesterol, the same method confirmed LDL’s known harm, so the method itself was working.
People with higher HDL have fewer heart attacks. Each standard deviation higher goes with 38% lower odds. The statistics are sound.
Every step in the HDL story used correct statistics. The mistake was the inference from “goes with fewer heart attacks” to “raising it prevents heart attacks.” An agent that can run all of these analyses has the method. Knowing which conclusion each one can support is the judgment, and TheoPhysEval tested only the method.
Expertise has a range
The philosopher Nathan Ballantyne named the idea in a 2019 paper in Mind and in his book Knowing Our Limits. The paper’s abstract begins plainly. “Epistemic trespassers judge matters outside their field of expertise.”
Ballantyne’s term for judging a question with the evidence and skills of only some of the fields it needs. He argues that once we learn we have trespassed, we should often hold our view with less confidence. The charge concerns how a belief was reached. A trespasser can still turn out to be right.
One of Ballantyne’s examples is Linus Pauling, who won the 1954 Nobel Prize in Chemistry for work on the chemical bond. In the 1970s he championed high doses of vitamin C, first against colds and then against cancer. The Mayo Clinic tested the cancer claim in 1979 and again in 1985, giving patients with advanced cancer 10 grams a day or a placebo. Neither trial found any benefit.
Pauling’s chemistry was first-rate, and it gave him no special access to what a vitamin does for cancer patients. His reputation still made the claim travel.
The philosopher Jamie Carlin Watson replied in 2022 that crossing is warranted more often than Ballantyne allows. Watson calls overlapping fields epistemic neighbours. Their experts “have authority to speak in those other domains to the degree that their competence in their own domain constitutes competence in the neighboring domain.” Read together, the two papers give a working test for any crossing. The question is how far the crosser’s competence reaches into the new field, and the new field’s own evidence answers it.
A haematologist gives an opinion on a cancer patient’s treatment plan.
Neighbour. Watson’s example. The fields overlap so much that neither specialist distrusts the other.
A family physician tells a patient that a vaccine is safe.
Neighbour. Also Watson’s. Vaccine advice rests on immunology and epidemiology, close enough to a physician’s training.
A Nobel-winning chemist promotes vitamin C as a treatment for cancer.
Trespasser. Ballantyne’s example, Linus Pauling. Two trials found no benefit.
A geneticist asks an AI agent for a drug target and presents the agent’s case as their own judgment.
Open question. Ballantyne wrote about people. The next section takes up the machine.
AI removes the friction that used to warn us
Crossing into a new field used to take months of reading. The slowness did a job, because researchers ran into the gaps in their knowledge before they reached a confident conclusion. An AI agent removes most of that friction. A reader can now follow every sentence of an AI-written argument and still be unable to tell whether it is valid.
Before AI
Entering a new field took months. On the way, you fell into the gaps in what you knew, and you noticed them.
With an AI agent
An agent carries you to a confident conclusion in minutes. The gaps are still on the road you skipped.
Take a hypothetical geneticist who asks an AI agent for a drug target. The agent combines genetic association studies with protein measurements from blood and builds a persuasive case that blocking one protein would lower the risk of a disease. Whether the case holds turns on questions that belong to different experts.
- Cause. An association can exist without the protein causing the disease, as the HDL story showed.
- Tissue. A protein measured in blood may behave differently in the organ where the disease starts.
- Side effects. A drug can do harm elsewhere in the body, as torcetrapib did.
A conclusion can also be right for bad reasons. If a target chosen through faulty reasoning later works, the same reasoning will be trusted on the next target, where it may fail.
Ballantyne wrote about people who trade on standing earned in one field to speak with authority in another. An AI agent has no standing of its own. The person who presents its conclusion as their own evaluated judgment borrows authority twice, from their own field and from the machine. People can now acquire conclusions faster than they acquire the ability to evaluate them.
Testing becomes the bottleneck
Clark asked his audience to picture science at a new scale. “What happens when instead of ~10 million scientists, we have a billion virtual scientists directed by our own ten million?” He also named the catch. “An avalanche of new proposed drugs is of little use if only a handful can ever be tested.”
Checking costs differ by field. A physics answer can often be checked with a computer run. A drug can need a trial of 15,067 patients. A second bottleneck sits in front of the first, because a lab with a fixed budget has to choose which ideas to test. In When the answer is cheap, I argued that as answers get cheap, the premium moves to choosing the questions. Figure 7 is a toy model of that choice.
Ideas tested
True among tested
2%
Discoveries a year
2
With 10,000 ideas a year and random picks, the lab tests 1% of them and makes about 2 discoveries a year, the same number it makes with 100 ideas.
In the toy, a lab that picks at random makes two discoveries a year whether it receives 100 ideas or 10,000. More ideas pay off only when the picks get better. Picking well is what the TheoPhysEval authors called taste, and whether AI systems will develop it is an open question. Exams mostly test the skill the agents already have in abundance, solving a well-posed problem. They rarely test choosing one.
How a method from economics learned genetics
Some methods have crossed fields and held up. Mendelian randomization, the method behind the genetic half of the HDL story, is one of them, and it started with butter.
In 1928, the economist Philip Wright wanted to separate supply from demand for butter and flaxseed. In an appendix to a book on tariffs, he used outside factors, such as the weather, that move one curve and leave the other in place. Economists now call this method instrumental variables.
In 1986, the nutrition scientist Martijn Katan proposed the same logic for genes. Low cholesterol had been linked to cancer, and nobody could tell which caused which. Variants of the APOE gene shift cholesterol levels from conception, so comparing people by APOE type could settle it. George Davey Smith and Shah Ebrahim named the approach Mendelian randomization in 2003.
The method crossed over intact, and its assumptions had to be argued again. An instrument must affect the outcome only through the variable being studied. A gene often has several effects, which geneticists call pleiotropy, so they built tools such as MR-Egger regression to test for a second path.
AI could make crossings like this far more common. Turning them into reliable science still runs through the destination field’s evidence.
Crossing boundaries without losing our standards
Clark’s vision goes after one of the oldest limits on research, the small number of people able to work on hard problems, and TheoPhysEval suggests that limit is already loosening in theoretical physics. Working in a field and holding the authority to draw conclusions in it are separate achievements. AI may in time manage both. The way to find out is to test it field by field, as the physicists tested calculation.
The physicists could grade Astra because every problem came with an answer key, and the key itself had errors until the agent pushed back. Most questions in biology and medicine have no key yet. For HDL, writing one took a trial of 15,067 patients and a genetic study of more than 116,000 people.
Further reading
- Jack Clark, Inside the Event Horizon (Per Jacobsson Foundation Lecture, IMF, 17 October 2026), and the text of the speech.
- Chandra et al., TheoPhysEval: Assessing AI agents at autonomous theoretical physics problem solving (arXiv, 2026).
- Nathan Ballantyne, Epistemic trespassing (Mind, 2019).
- Jamie Carlin Watson, Epistemic neighbors: trespassing and the range of expert authority (Synthese, 2022).
- Voight et al., Plasma HDL cholesterol and risk of myocardial infarction: a mendelian randomisation study (The Lancet, 2012).