Key points
- METR, an AI-evaluation nonprofit, spent 11 weeks inside Anthropic and estimates its model progress runs about twice as fast as a year ago, with a stated range of 1.3x to 3x.
- METR attributes roughly 80 percent of the speedup to AI use in Anthropic's own research, and says it is confident compute growth does not explain it.
- Anthropic redacted most of the evidence METR rated most informative; cofounder Sam McCandlish said sharing it risks handing out a roadmap for others.
- Anthropic plans to publish its own report on AI progress speed in the coming weeks, per McCandlish.
Why it matters for marketers: The vendors behind your marketing tools may be improving faster than public releases show, so plan for rapid capability jumps and thinner transparency about what is coming.
The AI-evaluation nonprofit METR published a report on Thursday after 11 weeks of access to Anthropic's internal data and staff interviews. It estimates the company's model improvement is running about twice as fast as a year earlier, with a plausible range of 1.3x to 3x. According to METR, about 80 percent of that acceleration comes from AI being used inside Anthropic's research, and the auditors are confident it is not simply a result of more training compute.
For now the gains come mainly from AI assisting human researchers rather than working on its own, though METR sees signs that autonomous contributions could grow. It found no clear bottlenecks that would slow the pace. Employees also reported feeling roughly 12 times more productive with the latest internal model, a figure METR treats as loose evidence that likely overstates real gains. The report also covers three models that finished training between July and October and have not been released.
The transparency problem is central. Anthropic had the right to redact the report and removed most of the evidence METR considered most informative. McCandlish acknowledged this makes METR's work harder to check, but said the details are commercially sensitive and could help others speed up. Critics, including AI 2027 author Daniel Kokotajlo, have called the stance hypocritical given Anthropic's public calls for caution. Anthropic declined The Verge's questions beyond pointing to McCandlish's post.
For marketers and content teams that build on Claude or similar tools, the practical takeaway is that the models you can use today may trail what vendors run internally, and capability steps could arrive faster than planned. Review your tool contracts and workflows for flexibility. Watch for Anthropic's own report on the speed of AI progress, which McCandlish said is due in the coming weeks, and for any policy response.
Sources
This brief is our own summary. Read the full reporting at the source.
- The Verge: Why a new report on Anthropic automating AI development is alarming researchers
- The Information: Anthropic’s R&D Speed May Have Doubled Due to AI, Researchers Find
- The Decoder: METR finds Anthropic may have crossed its own AI acceleration warning threshold back in July
- The Rundown AI: An inside look at Anthropic’s AI speedup