StateBench tests how language models track changing states in long documents

Key points

• Benchmark has four difficulty levels, from fixed values to swap-based reassignments, and tests the same problem in several formats.
• Frontier models including Gemini 3.8 Flash and GPT-6 Sol performed well on simple levels but fell sharply on harder ones.
• On the longest general finite problems, frontier model accuracy dropped to 13% or lower as inputs grew.
• Small Gated DeltaNet and hybrid models trained directly on StateBench stayed near perfect far beyond training lengths, while transformers never learned the hardest level.

Why it matters for marketers: Content teams using long-document AI should test models on the state-tracking level their workflows need, since frontier models still degrade on harder cases.