Anthropic is developing AI twice as fast as a year ago, review finds

Researchers suspect AI improving itself is the main driver of the speedup

Anthropic is improving its artificial intelligence models about twice as fast as it was a year ago, according to an analysis released Thursday by an independent monitoring group that the San Francisco company invited to examine its internal data. The assessment comes as Anthropic prepares to go public later this month in what could be the biggest initial public offering of stock ever.

Model Evaluation and Threat Research, a Berkeley, California-based nonprofit known as METR, estimated that Anthropic is improving the capabilities of its models to do “well specified tasks” around twice as fast as it was a year earlier. The nonprofit’s researchers said they believed much of the reason for the faster development is the help of AI itself. Anthropic has said its Claude chatbot increasingly is helping the company develop newer, more advanced versions of itself.

Anthropic invited the METR researchers to review its internal data and interview its staff as it prepares for its Wall Street debut, amid debate over the potential for AI models to elude human control with catastrophic consequences. Anthropic did not immediately respond to messages seeking comment.

The findings are significant because details about the technology — and its risks — have been provided primarily by AI companies, which have been encouraged by President Donald Trump’s administration to police themselves on matters of safety. METR is funded through donations and does not accept compensation for its work, though AI companies it has partnered with for evaluations have provided access and tokens, which the group uses for evaluations, research and engineering.

One of METR’s two reports was based on analysis of internal data provided by Anthropic over a period of about 11 weeks, from mid-August through late October, as well as employee interviews. The group said its assessment focused on two questions: whether the growth rate of Anthropic’s internal capabilities has accelerated, and how much of that was attributable to AI. METR said it was given “unprecedented access to internal data” on Anthropic’s AI research and development process.

Anthropic’s redactions to the report limited how METR could communicate more detailed estimates of the acceleration, and the group reported some internal disagreements and uncertainty within its team. Even so, METR said it believed the company’s rate of improvement had doubled. “Anthropic appears to have made as much progress on benchmarks in the past year as they did between October 2023 and October 2025,” according to the METR report.

The report was peppered with caveats. METR said it could be overstating its findings, because the acceleration might only be limited to “well-specified tasks” that can be graded automatically or with a language model, instead of “messy” tasks that are harder to define. It also could be that Anthropic understated its previous progress, making the latest number look bigger, the group said.

METR said it was able to “confidently” rule out the growth of computing power used in training AI models as the reason for the acceleration. The amount of computing power, known as compute, is traditionally directly related to how well an AI model performs. The researchers said their guess was that the improvement was down to using AI itself to improve AI.

A key concern, METR said, is that using AI to improve itself could start to compound the acceleration — a prospect the group said was especially worrying given that researchers found no “bottlenecks,” such as restrictions on computing power or data, holding back Anthropic’s AI progress.

“We understand that AI is not yet able to make highly significant and fully autonomous contributions to the most important parts of the model training process,” the report said. However, METR said Anthropic researchers described significant “signs of life” implying that could happen soon.

It was during the research group’s assessment period that Anthropic CEO Dario Amodei called for the AI industry to slow its work so safety measures can catch up. In a Sept. 12 essay responding to mounting concerns about runaway AI, he urged the AI world to “slow the pace at which we improve the capabilities of AI models.”

In that essay, Amodei wrote that AI advances have been driven primarily by the concept of recursive self-improvement, which he defined as “AI’s growing ability to build the next generation of AI.” “Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all,” Amodei wrote.

Amodei also used the essay to champion the idea of having external evaluators embedded within the leading AI labs, specifically highlighting METR as an appropriate third-party evaluator. OpenAI CEO Sam Altman said at the time that he also supported the idea of independent evaluators with employee-like access. METR has conducted evaluations of model capabilities for Anthropic, OpenAI and other labs for years.