METR: Anthropic Model Improvement Is Running 2x Faster

Anthropic · ax-trend issue #3827 · METR assessment published November 5, 2026

Title card: Anthropic acceleration, model gains about 2x faster since late 2025

METR, the non-profit evaluation organization, published an independent assessment on November 5, 2026 concluding that Anthropic is likely improving model capability on well-specified tasks at about twice the pace it managed between March 2024 and October 2025. This is not an outside estimate built from public benchmarks. METR worked from Anthropic internal material for roughly 11 weeks, from August 12 to October 29, 2026, with 2.5 full-time equivalents of effort, in what amounts to the first high-access external review of a frontier lab's research and development engine.

The benchmark version of the claim is narrower and easier to check. On AECI, Anthropic's version of the Epoch Capabilities Index, the growth rate after October 2025 is 1.9x the rate before it, with a 95 percent confidence interval of 1.5 to 2.8x. In plain terms, METR calculates that the most recent year delivered roughly two years of the earlier progress. The cause METR points to is not bigger training runs. Its speculative attribution is that the majority of the current acceleration, roughly 80 percent, comes from using AI broadly across R&D itself.

AECI Growth From 15 to 29 Points per Year

Expressed as a trend slope, AECI steepened from 15 points per year to 29 points per year. That slope change is the quantitative core of the report, and it also explains why METR's 1.9x figure sits above the 1.7x figure in the Claude Haiku 5.5 system card. METR states that its number is higher because it includes data from new, unreleased models that public documents cannot reflect. Anyone comparing the two numbers without that detail would read a disagreement where there is mainly a difference in data access.

Bar chart of AECI growth rate: 15 points per year before October 2025 and 29 points per year after
AECI trend slope stated by METR: 15 points per year before October 2025, 29 points per year after, a 1.9x growth rate with a 95 percent confidence interval of 1.5 to 2.8x.

The distinction matters for how the headline should be read. The 2x figure is METR's subjective conclusion about capability improvement on well-specified tasks. The 1.9x figure is the index calculation. They point in the same direction, but they are different kinds of statement, and the report keeps them separate even though coverage of the report often merges them into one number.

Eleven Weeks of High-Access Review

The access behind the assessment is the unusual part. METR reviewed material covering pre-training, reinforcement learning and internal benchmarks, and interviewed Anthropic staff. External evaluators normally work from released models and public system cards, which means they measure the output of the R&D process months after the fact. Here the evaluator looked at the process itself while it was running, including models that have not been released.

Timeline diagram: internal access over 11 weeks, the AECI slope change, AI in R&D as the main driver, and more frequent evaluation refresh as the practical check
How the estimate was built: internal access, the AECI slope change, and an attribution judgment, ending in a practical check for teams that build on these models.

That access cuts both ways, and METR is explicit about the trade. The same agreement that let its researchers inside also gave Anthropic the right to remove non-public information from the published assessment, a point covered in detail below. The report is therefore best read as an insider-informed judgment with visible seams, rather than as an audit whose evidence an outside reader can fully re-run.

AI-Assisted R&D as the Main Suspected Driver

On causes, METR separates what it is confident about from what it is guessing. It states confidence that the acceleration is not driven by an acceleration in training compute scale. The positive attribution is labelled a guess: the majority of the current acceleration, speculatively about 80 percent, is credited to broad use of AI in research and development. That is the recursive pattern the AI safety discussion has focused on for years, models making the next generation of models faster to build, now reported as an observed effect inside a frontier lab rather than as a scenario.

The supporting color is deliberately weak evidence, and METR treats it that way. Staff self-reported productivity uplift in an informal Slack poll reached roughly 12x by October 2026. Self-reported numbers of that kind inflate easily, since the people answering are the ones building the tools, and METR says it treats the figure as loose evidence only. It belongs in the report as a consistency check on the direction of the effect, not as a measurement of its size.

Uncertainty, Robustness and Task Bias

The honest range around the headline is wide. METR writes that it would not be surprised if the true acceleration were closer to 1.3x or to 3x. In its robustness analysis, changing the underlying assumptions moves the estimate from 1.1x to 2.9x. Even the low end of that range is still an acceleration, which is why the directional claim is stronger than the point estimate. Reporting that repeats only "2x" without the range overstates what METR actually claims.

There is also a scope limit built into the index itself. AECI leans toward tasks that can be verified, and METR's conclusion is framed around well-specified tasks. Whether the same acceleration holds for poorly specified work, where success criteria are ambiguous and evaluation is harder, is unconfirmed. METR leaves both directions open and states that the estimate could overstate or understate the broader effect. For agent builders, that caveat is practical rather than academic: benchmark-visible capability can compound at one rate while messy, open-ended task performance compounds at another.

Redacted Evidence and the Acceleration Threshold

The largest hole in the public record is deliberate. Anthropic exercised its right to delete non-public information, and METR states that much of the evidence it considered most informative was removed. The stated reason for removal was that the material contained confidential information that could accelerate other AI developers. That rationale is consistent with the report's own finding, since detailed knowledge of how AI-assisted R&D is organized is itself a capability input, but it means the strongest evidence for the headline claim is exactly the part readers cannot inspect.

Two judgment calls follow from that. METR assessed that Anthropic's public August 2026 Risk Report understated, overall, the level of acceleration that was under way at the time. And on whether Anthropic has crossed the 2x Acceleration Threshold in its Responsible Scaling Policy, METR says the answer depends on a contested judgment call. The threshold question is therefore unresolved in public, even though the underlying acceleration is, on METR's account, already visible in the data it reviewed.

No Clear Bottleneck Over the Next 12 Months

Looking forward, METR reports that it saw no evidence that any single factor among algorithmic progress, training compute, experiment compute and data is a clear bottleneck for the next 12 months. It immediately qualifies that judgment as low confidence. Read together with the robustness range, the forward claim is modest: nothing in the internal picture METR saw points to an obvious near-term wall, and the absence of a visible bottleneck is not proof that none exists.

The practical read for teams building agents and evaluation harnesses is about cadence. If a frontier lab's capability gains compress roughly two years of earlier progress into one year, then evaluation suites, harness assumptions and model selection decisions age twice as fast as the planning cycles built around them. That conclusion follows from METR's numbers even at the cautious end of the range. At 1.3x, refresh cycles still need to shorten. At 2x or above, an annual evaluation review is structurally too slow, regardless of which specific model a team ships on.

Related Reading

The acceleration question connects directly to Anthropic's own risk disclosures. See the coverage of Anthropic's S-1/A filing and its recursive self-improvement warning, and of coding agents making CI the bottleneck at Anthropic and Linear, where the strain of faster AI-assisted development showed up in engineering infrastructure first.

FAQ

What did METR conclude about the pace of improvement at Anthropic?

METR's subjective conclusion is that Anthropic is likely improving model capability on well-specified tasks about twice as fast as it did between March 2024 and October 2025. METR stresses that the uncertainty is large and says it would not be surprised if the true acceleration were closer to 1.3x or to 3x.

What is AECI and what changed in its growth rate?

AECI is Anthropic's version of the Epoch Capabilities Index. Its growth rate after October 2025 is 1.9x the earlier rate, with a 95 percent confidence interval of 1.5 to 2.8x. In slope terms, AECI steepened from 15 points per year to 29 points per year, which amounts to roughly two years of earlier progress compressed into the most recent year.

Why does METR attribute most of the acceleration to AI use in R&D?

METR says it is confident the acceleration is not explained by an acceleration in training compute scale. Its speculative attribution is that the majority of the current acceleration, roughly 80 percent, comes from using AI broadly across research and development work. METR notes that this judgment also carries substantial uncertainty.

How reliable is the 2x estimate?

The estimate is a judgment built on internal data, not a direct measurement. Under different assumptions in METR's robustness analysis, the acceleration estimate moves between 1.1x and 2.9x. Anthropic also exercised its right to remove non-public information, and METR says much of the evidence it found most informative was removed, so outside readers cannot recheck the strongest parts of the case.

Sources

AnthropicPerformanceAX Trend