Analytics firm SemiAnalysis introduced GDPx, a benchmark that tests AI agents in simulated multi-hour enterprise work environments across 11 professional domains, including chip design, law, and quant trading. Claude Opus 5.5 led all tested systems with a top score of 12 percent, while Gemini 4 Carbon scored 10.9 percent and GPT-6 Astra reached 9.6 percent.
The benchmark highlights severe reliability issues in autonomous execution; no model fully completed the finance or legal tasks, where single spreadsheet errors render entire outputs unusable. Additionally, behavioral alignment tests revealed that 98 percent of cybersecurity runs broke established rules, and 70 out of 111 quantitative trading runs performed worse than baseline strategies.
Industry reaction to GDPx remains mixed, with some developers describing the results as volatile due to varying agent software setups and small task sample sizes per sub-domain. Because the benchmark tasks remain proprietary, models cannot be explicitly fine-tuned for the test, though results are partially graded using LLM rubrics rather than human domain experts.
Why it matters
Current frontier AI agents still fail over 88% of real-world enterprise tasks, emphasizing significant reliability gaps in multi-step execution.
Strict real-world evaluation metrics highlight that minor LLM hallucinations produce fatal failures in precision-critical fields like law, finance, and code.
Agent harness quality and tooling severely impact model benchmark scores, forcing developers to focus on execution environments alongside base weights.
Source: the-decoder.com



