Agents improve LLM data strategies, with up to 13.34% AIME25 gains

Key points

• Agentic Data Engineering framework lets agents pursue, revise, or combine research directions using cumulative findings.
• Selected strategies reportedly gained up to 13.34% on AIME25 Pass@32 and 5.85% averaged across four held-out benchmarks.
• Tasks covered math and code data selection, math reward design, and math curriculum learning under fixed training conditions.
• Later in-loop score gains did not always improve held-out generalization, according to the authors' trajectory analysis.

Why it matters for marketers: Marketers and content teams using LLMs should note that in-loop gains may not carry over to held-out performance, so validate results on separate benchmarks.