What changed in AI today

• Jose Hernandez-Orallo proposes a test based on Levin's Kt, with LLMs as the representational machines.
• Difficulty is measured on a ratio scale in bits, so fresh items can be made before humans or frontier models solve them.
• A first battery of certified items was administered to twelve LLMs, per the preprint's abstract.
• Reported curves decrease with difficulty into a high-difficulty region the author calls the supra-intelligence window.

Why it matters for marketers: Content teams testing AI benchmarks may find this a method for generating fresh, certified items that resist saturation, though the work is an unreviewed preprint.

• Study of 220 data science tasks found most agentic solutions contain at least one failure mode.
• Authors report a positive correlation between data leakage failures and predictive performance.
• A reusable test suite checks six specific failures, including data leakage and inappropriate validation.
• Agents reportedly lack problem solving creativity and domain knowledge compared with competition-winning data scientists.

Why it matters for marketers: Content and analytics teams relying on AI data science agents should audit outputs for leakage and validation errors, not just accuracy scores.

• Framework needs no external teacher model, using a loop of task synthesis, training, and diagnosis feedback.
• Qwen3.6-27B used as loop model; reported gains on SkillsBench, Claw-Eval-Live, and ClawGym-Bench.
• Refined tasks reportedly transfer to smaller Qwen3.5-4B and 9B models, improving results across all benchmarks.
• Preprint submitted October 9, 2026 on arXiv; the paper has not been peer reviewed.

Why it matters for marketers: Content and AI teams training agents may get a path to improve agents without relying on a stronger external model to generate training data.

• Method adds a corrector that refines the base denoiser's clean-sample prediction at each sampling step.
• Tested on DiT, SiT and JiT; reportedly improves FID for unguided and unconditional generation.
• Reported to be competitive with classifier-free guidance, and combining both improves diversity and reduces oversaturation.
• Corrections can be distilled back into the base model, keeping quality gains without extra inference cost, per the authors.

Why it matters for marketers: Teams running diffusion models without guidance may get better samples from a correction stage, and distillation could avoid added inference cost.

• Extends the original benchmark, keeping its 46 tasks, correcting defects, and extending 31 tasks across 13 domains.
• Evaluation horizon raised from 90 minutes to three hours, with verifier checkpoints recorded every 30 minutes.
• On 30 reconstruction tasks, binary verifier scores averaged 0.221 versus 0.653 for dense progress ladder scores.
• Across 1,145 scored runs from 15 frontier models, GPT-6-astra led with dense mean reward 0.69.

Why it matters for marketers: Teams evaluating agents on multi-hour workflows can use partial-progress scoring, since outcome-only checks reportedly miss much intermediate progress.

• Paper arXiv:2610.14149 introduces VETNET, an ecosystem of live benchmark datasets and evaluation frameworks for animal science.
• Submitted October 9, 2026, by Katherine Koebel with 35 other co-authors from academic and non-academic sectors.
• Aims to close a gap between AI progress in human medicine and fragmented veterinary AI development, per the abstract.

Why it matters for marketers: Content and data teams in animal health may gain a shared standard for judging veterinary AI tools, though the source describes a framework, not a released product.

• Study evaluated 26 models, 19 open-weight and 7 closed, on the CTI-REALM security benchmark.
• An open-weight model reportedly matched the strongest closed models at under a tenth of their cost, given enough turns.
• A published limit on agent messages reportedly zeroed out tasks, so relaxing it changed rankings and favored open-weight models.
• Different models lead different workflow stages, suggesting per-stage routing instead of choosing one model by overall score.

Why it matters for marketers: Teams building security agents can weigh routing workflow stages to different models, and should not judge models by price or a single overall score.

• Researchers applied counterfactual interventions to a Proximal Policy Optimization agent trained in the Crafter environment.
• The value function reportedly learned rules such as when a diamond is collectible, and that water and food affect health.
• The paper also reports heuristics about inventory, and discusses safety implications and limits of current reinforcement learning agents.
• Submitted to arXiv on 9 October 2026; the paper is 20 pages with 15 figures and 3 tables.

Why it matters for marketers: Teams building agent-based content or simulation tools may need to weigh whether reward-trained agents internally model environment rules, which affects safety review.

• Researchers introduce MOSAIC, a method that attributes vision-language model predictions to training influence at the level of image regions and text phrases.
• Tested on Qwen and Gemma models across captioning, chest X-ray understanding, and visual question answering tasks.
• Per the paper, ablating influence-ranked latents changes model outputs more than ablating latents ranked by activation or frequency.
• Submitted to arXiv on 9 October 2026 as a preprint, listed under cs.AI.

Why it matters for marketers: Teams auditing image-text model outputs gain a research method for tracing which visual evidence and training examples drive a given caption or answer.

• Proposed architecture: cloud model writes a program, which defers bounded questions over user data to a small local model via a function call.
• On AppWorld, success rate of a code-only agent improved in all six backbone and partition combinations tested, by up to 14.9 percentage points.
• Gains recover a substantial part of the gap to a cloud-hosted ReAct loop, according to the authors, while user data stays on the device.
• Paper was submitted October 9, 2026 and accepted at the SLM-Agents workshop at NeurIPS 2026.

Why it matters for marketers: Content and product teams building privacy-sensitive agents may be able to use small on-device models for run-time interpretation without sending user data to the cloud.