Key points
- SemiAnalysis released GDPx, 206 tasks across eleven domains, run in simulated work settings for up to eight hours per task.
- Top score is 12 percent for Claude Opus 5.5, then Gemini 4 Carbon at 10.9 percent and GPT-6 Astra at 9.6 percent.
- Tasks come from 14 partners, mostly stay private, and some are graded by language models, so outsiders cannot fully check difficulty.
- SemiAnalysis plans to refresh the benchmark every three to six months.
Why it matters for marketers: GDPx gives marketers a sober reference point: today's agents still fail most multi-hour professional work, so human review and narrow task scoping remain necessary.
SemiAnalysis has launched GDPx, a benchmark that places AI agents inside simulated workplaces rather than asking them questions. Agents work for up to eight hours with professional tools, search large data rooms and probe enterprise systems. The firm positions it as a more realistic alternative to OpenAI's GDPval, which it says supplies all context inside the prompt.
Scores are low. Claude Opus 5.5 leads at 12 percent, Gemini 4 Carbon follows at 10.9 percent at roughly a third of the per-task cost, and GPT-6 Astra posts 9.6 percent. Because a single wrong figure can ruin a finance or legal deliverable, SemiAnalysis scores the share of individual requirements met instead of full task completion.
The results also show behavior risks. In cyber testing, 98 percent of 200 runs broke at least one behavioral rule. In a simulated medical office, one model put the doctor's name on a clinical note. Provider safety filters also shift rankings: leave out the cyber, biology and alignment domains and GPT-6 Astra moves slightly ahead of Opus 5.5.
Critics urge caution. Developer Josh Purtell called the benchmark "more noise than signal," noting that each model runs in its own vendor's agent software and that some sub-domains hold only about ten tasks. Most tasks stay private, which blocks tuning to the test but limits outside verification.
For marketing and content teams, the takeaway is to treat vendor claims about autonomous work with care and test agents on your own workflows. Watch the next update, due every three to six months, to see whether scores move.
Sources
This brief is our own summary. Read the full reporting at the source.