The performance gap between open-weight AI models and proprietary frontier systems in cyber offensive tasks has narrowed to between four and seven months, down from six to ten months in 2025, according to evaluation data from Vals AI and the UK AI Security Institute (AISI). Testing on the Vals CyberOffense benchmark revealed that open-weight model Deepseek V4.1 Flash 1017 completed 72.8 percent of autonomous intrusion objectives at approximately $3 per run, approaching OpenAI’s restricted GPT-6 Sol model, which led with 80.8 percent at a cost of roughly $22.

The findings highlight significant cost disparities and execution traits between open and closed systems. In AISI cyber-range tests, a run using open-weight Deepseek V4-Pro cost $1.19 compared to $85 for Anthropic’s Opus 4.5. Researchers observed that open models required extended runtimes to achieve high success rates, with significant progress occurring only after processing over 100 million tokens.

Security researchers emphasize dual-use implications, noting that while public weights prevent central usage restrictions, the same capabilities enable defensive teams to automatedly identify and patch vulnerabilities prior to exploitation. Additional research cited from Anthropic confirmed that accessible models like GLM-5-3 can generate software exploits at capability levels approaching internal top-tier models.

Why it matters

  • Open-weight models provide low-cost automated security testing capabilities for enterprise defense.

  • Rapid narrowing of open vs closed model gap limits the effectiveness of centralized safety guardrails.

  • Long-horizon execution time is becoming a key variable in evaluating agent threat models.

Source: the-decoder.com