Home / Blog

AI research

How much is AI speeding up AI research?

For eleven weeks, a team from the nonprofit METR worked through Anthropic’s internal data to measure how fast its AI models are improving. Their best guess is that, on tasks with clear right answers, the models now improve about twice as fast as they did before October 2025. They think most of the extra speed comes from Anthropic’s staff using AI in their research. Anthropic redacted more than half of the evidence METR found most informative.

Main takeaways
  • The benchmark trend bent in October 2025. Before then, Anthropic’s models gained about 15 points a year on a score that combines many AI tests. After it, they gained about 29, which is 1.9 times the old speed.
  • More computing power does not explain it. METR is confident the speed-up did not come from faster growth in the computing power used for training. Its guess is that about 80% of the extra speed in October 2026 came from AI.
  • Staff polls put AI’s help at 12 times. The benchmark speed-up is 1.9. A one-time boost lifts progress once, so what matters to METR is how fast the help keeps growing.
  • The answer depends on analysis choices. One assumption about how hard benchmark questions are moves it anywhere from 1.1 to 2.9. Public models alone, from every company, show almost no acceleration.
  • Readers have to lean on METR’s judgment. By METR’s own count, the public report describes the results of 1 of the 7 kinds of evidence it reviewed. METR wrote that the public must rely on its judgment “much more than would be ideal.”

“It is hard to see how I am not being 3x more productive,” a researcher at Anthropic told an interviewer. “I feel like I have postdocs and grad students, and we are working towards a larger project.”

The interviewer worked for METR, a nonprofit that studies whether the companies building the most advanced AI agents are on track to lose control of them. An agent is a program that uses an AI model to carry out tasks by itself. Anthropic, which makes the Claude models, is one of those companies. From 12 August to 29 October 2026, a METR team worked on Anthropic laptops with what it calls “unprecedented access” to internal data. Its question was whether AI is now speeding up the work of building AI, and it published its answer on 5 November.

If AI is speeding up AI research, a better model helps build the next one sooner, and that one helps build the one after it sooner still. METR’s report warns that this “could lead to a sudden emergence of extreme and risky capabilities.” I wrote about the loop in What happens when AI starts improving itself?, and at the time I could find no outside check of the companies’ own numbers. This report is one.

From October 2023 to October 2025, Anthropic’s models went from Claude 2.1 to Claude Sonnet 4.5. How long did the same amount of progress take next?

Two years of progress, then the same amount in one year A timeline from October 2023 to October 2027. The first bar covers the two years from October 2023 to October 2025, Claude 2.1 to Claude Sonnet 4.5. The second bar covers the same amount of benchmark progress, from October 2025 to about October 2026. Oct 2023 Oct 2024 Oct 2025 Oct 2026 Oct 2027 Claude 2.1 → Claude Sonnet 4.5 2 years The same amount of progress again about 1 year

About one year. On METR’s benchmark measure, Anthropic appears to have made as much progress from October 2025 to October 2026 as in the two years before.

Figure 1. Make a guess. METR made the comparison, writing that Anthropic “appears to have made as much progress on benchmarks in the past year as they did between October 2023 and October 2025 (roughly, Claude 2.1 to Sonnet 4.5).” The robot is a cartoon.

Two years of progress in one year

METR’s main evidence is a single score that combines many tests, which the next section explains. Before October 2025, Anthropic’s models gained about 15 points a year on it. After that they gained about 29.

Acceleration

Acceleration is the new speed of progress divided by the old speed. An acceleration of 2 means two years of old progress now happen in one year. For a score that climbs a steady number of points each year, it is the ratio of points per year, so 29 divided by 15 gives about 1.9. For anything that multiplies, METR compares growth over the same span of time. A quantity that used to triple every year and now grows ninefold has an acceleration of 2, because tripling twice gives nine.

Most of the METR staff on the project believe that Anthropic is “likely improving the capabilities of their models on well-specified tasks at around twice the rate” of March 2024 to October 2025, and they “would not be surprised” if the true figure were closer to 1.3 or 3. A well-specified task has a clear answer that a program or another AI model can grade.

One score for many tests

A benchmark is a fixed set of test questions with known answers, used to score AI models. Benchmarks fill up. Once a model answers nearly every question correctly, a better model has no room left to show it. The research group Epoch AI built a way around this called the Epoch Capabilities Index, or ECI. It places models and benchmarks on one scale, so a model from 2024 and a model from 2026 can be compared even if they never sat the same test.

Each model gets one capability number. Each benchmark gets a difficulty number, plus a second number for how sharply scores on it rise. A model’s predicted score on a benchmark follows an S-shaped curve of the gap between its capability and the benchmark’s difficulty, and it reaches 50% where the two are equal. Anthropic runs its own version, AECI (Anthropic’s ECI), on a mix of its internal benchmarks and public ones, and METR recomputed it with Epoch’s public code. The numbers have no natural unit. Anthropic pins the scale by setting Claude 3.5 Sonnet, from June 2024, at 130.

Capability 148
Expected scores on three benchmarks of different difficulty Three S-shaped curves show the predicted score on an easy, a medium and a hard benchmark as model capability rises from 100 to 200. A dashed vertical line marks the chosen model. At a capability of 148 the easy test is nearly full and the hard test is near zero, so only the medium test separates nearby models. above 90%: nearly fullbelow 10%: near zero0%25%50%75%100%100125150175200Model capability (AECI)Predicted scoreEasy testMedium testHard test
Easy test94%
Medium test43%
Hard test4%

Claude Sonnet 4.5 sits at 148. The easy test is nearly full and the hard test is still near zero. Only the medium test shows much difference between nearby models.

Figure 2. Drag the slider or press a model. The S-shaped curve is the assumption AECI makes. The three benchmarks and their difficulties are made up, and the model positions are real AECI scores read off METR’s chart.

A test tells nearby models apart only on the steep middle of its curve, which is why the index needs tests of many difficulties. Model 4, an internal model from late 2026, scores above 50% on all 15 benchmarks it was tested on. Nine months earlier, Claude Opus 4.6 still scored below 50% on many of its benchmarks. The authors write that this shortage of hard tests “makes us somewhat nervous about our ability to measure AECI now and in the future.” So far the estimate holds up. It stays at 1.9 when each benchmark stops counting once scores on it reach 85%.

The line bends in October 2025

METR plotted AECI for Anthropic’s models against the date each became available to most Anthropic employees. That date often comes before a public release. It also let METR include models that never left the company. Some are snapshots, saved versions of a model from during its development, and three are separate models that the report numbers Model 2 to Model 4.

Then it fit two straight lines that meet at one point, called a breakpoint, and let the data choose the breakpoint’s date. It landed on 7 October 2025.

If the old pace had held, Model 4 would score about 165. What did it score?

AECI of Anthropic models from early 2024 to October 2026 AECI of Anthropic models by the date most staff could use them. Claude 3 Opus scores about 125 in early 2024. The fitted line rises 15 points a year until 7 October 2025 and 29 points a year after. Model 4 scores 178.6 in late September 2026, about 14 points above where the old trend would have put it. 120130140150160170180Jan 2024JulJan 2025JulJan 2026JulAECIDate most Anthropic staff could use the modelpublicinternalpublic, not the best7 Oct 2025+15 a yearold pace continued+29 a yearModel 4: 178.6Claude Opus 4.5, 19 Nov 2025, AECI 152.4Claude Opus 4.6, 24 Jan 2026, AECI 157.3Mythos Preview, early snapshot, 24 Feb 2026, AECI 162.4Mythos Preview, 1 Apr 2026, AECI 165.7Claude Mythos 5, 1 Jun 2026, AECI 168.7Model 2, early snapshot, 10 Jul 2026, AECI 171.8Model 2, 7 Aug 2026, AECI 174.1Model 3, early snapshot, 9 Sep 2026, AECI 176.4Model 3, 22 Sep 2026, AECI 177.0Model 4, 25 Sep 2026, AECI 178.6Claude Opus 4.7, 10 Apr 2026, AECI 160.5Claude Opus 4.8, 19 May 2026, AECI 163.5Claude Opus 5, 17 Jul 2026, AECI 169.4Claude Mythos 5.1, 7 Aug 2026, AECI 172.6Claude Opus 5.5, 9 Sep 2026, AECI 174.7Claude 3 Opus, 29 Feb 2024, AECI 124.8Claude 3.5 Sonnet, 16 Jun 2024, AECI 130.1Claude 3.5 Sonnet (new), 10 Oct 2024, AECI 133.3Claude 3.7 Sonnet, 19 Feb 2025, AECI 141.8Claude Opus 4, 19 May 2025, AECI 144.9Claude Sonnet 4.5, 22 Sep 2025, AECI 147.8?

Hover over or tap a point to see the model.

Model 4 scored 178.6. After 7 October 2025 the fitted line climbs 29 points a year, against 15 before.

Figure 3. Make a guess, then hover over or tap the points. Blue dots are public models, gold diamonds are models and snapshots used only inside Anthropic, and open circles are public models that were not Anthropic’s best at the time. Points were read off METR’s chart by a script, and refitting them reproduces METR’s slopes of 15 and 29 points a year (15.4 and 29.3 in my refit) and its 7 October 2025 breakpoint.

Model 4 scored 178.6, about 14 points above the old line. The 95% confidence interval around the 1.9, the range of values the data are consistent with, runs from 1.5 to 2.8. It counts only the noise in each model’s score, and METR notes that it leaves out the uncertainty over which models go into the fit.

Anthropic publishes this kind of chart in its system cards, the technical reports released with each model. The card for Claude Haiku 5.5 shows a similar bend, fitted at September 2025, with an acceleration of 1.7. METR’s figure is higher mainly because it adds the newer models that Anthropic has not released.

A one-time jump would leave a different mark

A bend is one way to read these points. Anthropic’s earlier system cards raised another. Perhaps one model, Claude Mythos Preview, jumped ahead once, and progress then went back to its old speed.

METR checked which story better predicts models it had not yet used. It fit each story to the models up to Claude Mythos 5, from June 2026, and then compared each line with the five internal snapshots that came after. Points kept out of a fit like this are called held-out data. A line can match past points by luck, so predicting new ones is the harder test.

Two stories fitted through Claude Mythos 5, tested on five later models Anthropic models up to Claude Mythos 5 are used to fit two lines. The one-time jump line keeps a slope of about 16 points a year with a step of 6.5 points at Mythos Preview. The bend line rises from about 15 to about 32 points a year from late October 2025. The five later internal snapshots sit closer to the bend line. 120130140150160170180Jan 2024JulJan 2025JulJan 2026JulAECIDate most Anthropic staff could use the modelheld outpublicinternalbend: 15, then 32 a yearClaude 3 Opus, 29 Feb 2024, AECI 124.8Claude 3.5 Sonnet, 16 Jun 2024, AECI 130.1Claude 3.5 Sonnet (new), 10 Oct 2024, AECI 133.3Claude 3.7 Sonnet, 19 Feb 2025, AECI 141.8Claude Opus 4, 19 May 2025, AECI 144.9Claude Sonnet 4.5, 22 Sep 2025, AECI 147.8Claude Opus 4.5, 19 Nov 2025, AECI 152.4Claude Opus 4.6, 24 Jan 2026, AECI 157.3Mythos Preview, early snapshot, 24 Feb 2026, AECI 162.4Mythos Preview, 1 Apr 2026, AECI 165.7Claude Mythos 5, 1 Jun 2026, AECI 168.7Model 2, early snapshot, 10 Jul 2026, AECI 171.8Model 2, 7 Aug 2026, AECI 174.1Model 3, early snapshot, 9 Sep 2026, AECI 176.4Model 3, 22 Sep 2026, AECI 177.0Model 4, 25 Sep 2026, AECI 178.6

The bend line misses the five later models by a typical 1.9 points. The jump line misses by 3.9, about twice as much.

Figure 4. Pick a story, then show the later models. The red dashes are each prediction’s miss. I refit both lines to the points read off METR’s chart, and they reproduce the typical misses METR reports, 1.9 and 3.9 AECI points.

The bend missed a typical held-out model by 1.9 AECI points, and the jump by 3.9. METR repeated the test on many copies of the data, each made by drawing models at random from the real set with repeats allowed. This is called the bootstrap. The bend had the smaller miss in 81% of the copies. In METR’s reading, the newer models make a lasting change in slope look more likely than a single jump.

One assumption moves the answer from 1.1 to 2.9

METR ran 18 robustness checks, each repeating the estimate with one analysis choice changed. Most kept the answer between 1.9 and 2.2. Counting different snapshots of each model gave that range, and so did adding or dropping benchmarks.

One choice moved the answer a lot. The S-shaped curve is itself an assumption about how a benchmark’s questions are spread between easy and hard. If many benchmarks hold a small group of questions far easier than the rest, such as ones a model can guess, the acceleration falls to 1.1. A small group far harder than the rest pushes it to 2.9. Most versions fit the data about equally well, and METR writes that it has no “strong a priori reasons to prefer one.” It kept the standard curve because Epoch and Anthropic use it.

METR’s main estimate
Standard S-curve, all choices as published 1.9 [1.5–2.8]
Shape of the S-curve
Fewer very easy and very hard questions (probit) 1.8 [1.3–2.7]
A few questions far easier than the rest 1.1 [0.8–1.5]
A few questions far harder than the rest 2.9 [2.1–4.9]
More very easy and very hard questions (cauchit) 2.8 [2.0–4.1]
The mix learned from the data 2.6 [1.8–4.1]
Which snapshots of each model
Only the best snapshot of each model 1.9 [1.4–2.8]
Only the first snapshot of each model 2.2 [1.3–3.2]
A subset of models, reasons redacted 1.9 [1.4–3.3]
Every snapshot 1.9 [1.4–2.6]
Which benchmarks
Drop tests that use images 2.2 [1.7–3.5]
Only tests Claude labelled relevant to AI research 2.0 [1.5–2.9]
Drop tests that fall as models improve 1.9 [1.6–2.8]
Drop tests the S-curve fits badly 1.9 [1.4–2.6]
Stop using a test once it passes 85% 1.9 [1.4–2.3]
Penalty on extreme values
A 10 times weaker penalty 1.9 [1.4–2.8]
A 10 times stronger penalty 2.1 [1.6–3.0]
No penalty 1.9 [1.4–2.8]
A penalty toward a straight trend over time 1.9 [1.4–2.8]
Other data and estimates
Add other companies’ public models 1.4
Public models only, from every company 1.1
Epoch’s public index, public models only 0.99
Anthropic’s Haiku 5.5 system card 1.7
0.811.5234Acceleration, log scale

Every one of METR’s 18 checks shows some acceleration, from 1.1 to 2.9. Public models alone, from every company, show almost none.

Figure 5. Press a group to bring its rows forward. Each of METR’s checks changes one choice in its analysis and shows the estimate with its 95% confidence interval, from the tables in METR’s appendix. The scale is logarithmic, which gives each doubling the same width. The solid line marks 1, no speed-up, and the dashed line marks METR’s main estimate. Open circles have no published interval. The last group comes from METR’s comparison charts and Anthropic’s Haiku 5.5 system card.

Adding other companies’ public models gives a lower answer, 1.4. Before late 2025, models from companies such as OpenAI and Google had the top AECI scores most of the time, so they set the old trend. Anthropic’s models set the new one. One reading is that Anthropic was behind in 2025, so its own old trend was unusually slow and 1.4 is the fairer figure. The other is that rivals’ earlier models were tuned to do well on benchmarks, which makes their old trend too steep and leaves 1.9 closer.

Public models alone show almost no bend. Counting only released models from every company, dated by public release, the estimate is 1.09 on AECI and 0.99 on Epoch’s public index. METR traces most of the gap to two choices, looking at Anthropic’s models only and adding the ones it has not released. Summing up the benchmark evidence, METR wrote that it “would be surprised” to learn of no acceleration at all, or of one above 3.

METR credits about 80% of the extra speed to AI

A faster trend does not say what made it faster. Anthropic could have hired more researchers. It could also have spent more on compute, the computing power used to train each model. METR received what it calls “convincing evidence” that the speed-up did not come from faster growth in training compute.

The evidence itself was redacted.

On the other causes METR found no decisive evidence, and Anthropic’s own staff disagreed with each other. Two of the researchers interviewed named hiring. The team’s guess is that causes other than AI “likely played a large role” in late 2025, and that “perhaps ≈80%” of the extra speed by October 2026 comes from AI.

Interviews were one line of evidence. METR and Anthropic staff asked researchers how much AI had sped up progress in their own area. The notes then went to Claude, Anthropic’s model, with a request to sort each person by whether they supported an acceleration of 2 or more. Five of the 11 researchers supported it or leaned that way. Claude coded two as no and the other four as unclear. The model under study sorted the evidence about itself.

Does each researcher support an acceleration of 2 or more from AI? Sort them, then see how Claude sorted them.

“I would definitely endorse >2 years of progress in 1 year and I don’t think 4 is crazy.”

Researcher A

Claude’s call: yes. The same researcher also said, “IMO the bulk of the effect is recent models.”

“I’ve seen us discovering very promising directions ~entirely because people can spam 10 ideas at [agent-scaffold] and see what sticks.”

Researcher E

Claude’s call: leans yes. An agent scaffold is the software that lets a model run tasks by itself. The name of Anthropic’s was redacted.

“Claude lets researchers try many things quickly, most of which won’t work. …”

Researcher G

Claude’s call: unclear. In Anthropic’s notes, this researcher also named Claude and hiring as the two biggest contributors.

“I think it’s true [that] we’re now trying a lot more experiments at the same time because of claudification.”

Researcher H

Claude’s call: unclear. This is the researcher’s only quote in the table.

“IMO [hiring is] probably the biggest reason…”

Researcher K

Claude’s call: no. This is the researcher’s only quote in the table.

“Guesses 15-20% is due to AI. Don’t believe in self-reported productivity increase.”

Researcher L

Claude’s call: no. The same researcher also said, “One impact is SWE acceleration, which is extremely substantial.” SWE stands for software engineering.

Figure 6. Sort the quotes. All six are from the interview table in METR’s report, where square brackets mark words added or replaced in the report’s version of the notes. Each verdict is the label Claude gave that person after reading all of their quotes. The first button covers Claude’s yes and leans-yes labels.

METR’s best reading is that Anthropic now makes about two years of its old benchmark progress in one year, and that its staff’s use of AI is the main reason.

Staff polls put the speed-up at 12 times

Anthropic also polls its staff on its internal chat, where people vote with emoji reactions on how much the latest model speeds up their work. The average answer for Model 4 was about 12 times. METR calls this “only very loose evidence,” because the people who answer are not a random sample and the answers reflect “fairly low-effort consideration.”

Self-reports can run ahead of measurements. In a 2025 experiment by METR, 16 experienced open-source developers worked on 246 real tasks, each randomly assigned to allow AI tools or not. With AI they took 19% longer. Afterwards they believed AI had made them 20% faster.

Even an accurate 12 would not mean progress 12 times faster. Ideas get harder to find as the easy ones are used up. Doubling the density of computer chips now takes more than 18 times as many researchers as in the early 1970s, according to a 2020 study. So steady progress needs research effort that keeps growing. A fixed boost to every researcher raises the level of effort once, and progress steps up and then continues at its old pace. In METR’s words, “We thus care most about whether we see a large change in the growth rate of individual uplift, not the absolute level.” Uplift is METR’s word for how much AI speeds up one person’s work.

A one-time boost gives a step, growing help gives a bend Two panels share a time axis of six made-up years. The top panel shows AI help, which starts in year 3. The bottom panel shows progress. With help that doubles every year, progress climbs 30 units a year from year 3 instead of 15. AI HELP1x2x4x8xPROGRESS0306090120Year 0Year 1Year 2Year 3Year 4Year 5Year 6AI help arrives30 a yearwithout AI

Help that doubles every year adds a second doubling of effort each year. Progress climbs 30 units a year instead of 15, an acceleration of 2.

Figure 7. Press the buttons. Everything here is made up to show the logic. This toy calculation assumes research effort doubles every year without AI and that each doubling adds the same 15 units of progress, which is one way to model ideas getting harder to find.

The self-reports have grown fast, from 1.2 times for Claude 3.7 Sonnet in a targeted survey in early 2025 to 12 times for Model 4 in the chat polls. METR’s fitted trend has the reported speed-up multiplying by 3.9 each year. Growing help predicts a bend and a one-time boost predicts a step, the same two shapes METR compared in Figure 4. The bend predicted the later models better, though that test cannot say what caused it.

Nothing found that would slow it within a year

A bottleneck is a limited input that caps how fast the whole process can go. One researcher told METR they were “now only bottlenecked on ideas & compute, no longer bottlenecked on code.” METR looked for limits that could slow progress in the next 12 months, such as a shortage of computing power for experiments or of training data, and found none. The authors are not confident in this. Their view “changed several times as we got more detailed information,” and Anthropic redacted their analysis of several specific bottlenecks.

Having found no such limit, METR sees “a significant risk that further capabilities progress could lead to even faster acceleration over the coming year.” For now the speed-up comes from researchers using AI as a tool. The report says AI cannot yet make “highly significant and fully autonomous contributions” to the most important parts of training a model, though researchers described “signs of life.”

Most of the strongest evidence is blacked out

Under the contract, Anthropic could remove any non-public information from the public report. It “redacted a majority of the evidence we found most informative,” METR writes. Anthropic described the redactions as “aimed at removing confidential information that could accelerate other AI developers.”

Full report, about 100 pages

Benchmark scores of internal modelsAECI grew 1.9 times as fast after October 2025.Shared with METR
Key intermediate measuresShared with METR
Estimates of algorithmic progressShared with METR
Evidence of AI’s contributionStaff polls and interviews Shared with METR
Measurements of key research inputsMostly shared with METR
Bottlenecks and forecastsPartly shared with METR
Interviews with staffQuotes from 11 researchers Shared with METR

By METR’s own count, the public report describes the results of 1 of the 7 kinds of evidence, 2 in part and 4 not at all.

Figure 8. Switch the view. The seven kinds of evidence and their status come from the table in METR’s process section. The black bars stand for redactions, and their lengths are decoration.

METR filled in a checklist for outside evaluations, the AEF-1 from the AI Evaluator Forum. One item asks whether the evaluator shared enough of its methods for others to review the results, and METR answered “No.” In the report’s words, “the public has to rely on METR’s judgment much more than would be ideal.”

Other facts to weigh
  • The contract also covered a check of how Anthropic splits its computing power between making models more capable and safety work. Anthropic removed that part midway, before METR reached any conclusions, and said it would revisit it later.
  • METR took no payment. Anthropic supplied the laptops, plus free use of Claude, which METR spent mostly on data analysis.
  • Claude worked on both sides. METR says Claude was deeply involved in the majority of the data analysis it received and of its own analysis. Checking against the human interviews, METR found no sign that this skewed the results.
Where METR could be wrong

The estimate could be too low if growth is still speeding up, which some of the redacted evidence suggests. It could also be too low if benchmarks understate real effects. Epoch’s math scores have not sped up much. Yet in September 2026 an internal OpenAI system produced what OpenAI calls a solution to the Navier–Stokes problem, one of seven math problems that carry a $1 million Millennium Prize.

It would be too high if causes other than AI explain most of the bend. The speed-up might also hold only on well-specified tasks, and Anthropic’s models grew more slowly than the industry’s best before late 2025, so part of the bend may be a catch-up.

METR says that access at the level of an Anthropic employee, able to read documents on demand and sit in on internal debates, “would enable a significantly deeper assessment.” Its main measure has a second limit. Model 4 already scores above 50% on all 15 benchmarks it was tested on.

Further reading

AI research AI evaluation AI safety benchmarks