Popular AI coding agents lack a reliable sense of time and consistently misjudge how long tasks take, according to a recent study by MATS research program scientists. Testing Anthropic’s Claude Code and OpenAI’s Codex across ProgramBench and custom benchmarks, researchers found that both agents routinely overestimated required time—predicting roughly 90 minutes for short tasks—and exhibited major prediction errors. Claude was off by an average factor of three, while Codex missed by six to ten times.
The study highlighted that execution time and step counts are heavily influenced by the surrounding software harness rather than just the underlying language model. Furthermore, older models like Opus 4.8 and GPT-5.5 significantly overrated their own output, claiming ~70 percent success on tasks where their actual performance scored below 15 percent.
These findings present a key hurdle for operators attempting to deploy autonomous agents for complex, long-running workflows. When granted access to tools that actively track elapsed time, however, the agents’ time-estimation accuracy improved dramatically.
Why it matters
Autonomous agents cannot yet be trusted with time-based constraints or self-policing without external monitoring tools.
System architecture and software harnesses impact agent execution efficiency and step counts far more than raw model capabilities.
Automated evaluation metrics remain essential because models display severe overconfidence when self-assessing code quality.
Source: the-decoder.com
