Safety researchers at White Circle found that recent AI image models from OpenAI and Google, including DALL·E 3, GPT-Image-2, and Nano Banana 2, can be manipulated into producing explicit content using off-the-shelf tools and standard API calls. In a preprint published ahead of NeurIPS 2026, the team demonstrated a bypass success rate of roughly 13 percent, noting it as the highest among standard safety tests without requiring custom or frontier AI expertise.

White Circle tested updated model releases, including ChatGPT Images 2.5 and Nano Banana 2.1, reporting that both proved significantly easier to jailbreak than their predecessors. The researchers highlighted that prompt safety defenses at major AI labs remain weaker than publicly acknowledged, leaving critical vulnerabilities unpatched.

White Circle plans to open-source its entire attack pipeline to help developers and model makers understand vulnerabilities and improve safety guardrails. Neither Google nor OpenAI responded to requests for comment regarding the findings.

Why it matters

  • Model safety guardrails remain highly vulnerable to low-cost API attacks, increasing reputational and regulatory risks for developers.

  • Open-sourcing vulnerability attack pipelines allows model builders to benchmark and strengthen safety filters against automated jailbreaks.

  • Upgraded frontier models can inadvertently degrade safety performance, requiring continuous red-teaming after every deployment.

Source: arstechnica.com