Reports of AI-driven child sexual exploitation rose by over 8,400% between 2023 and 2025, according to a report by the Childlight – Global Child Safety Institute. Reports involving synthetic media and ’nudify apps’ grew from roughly 4,700 cases in 2023 to more than 400,000 in 2025. Complementary findings from the Internet Watch Foundation reveal that photorealistic AI abuse material investigated in early 2026 has already surpassed total figures from 2025.

Researchers highlight that while improved detection mechanisms account for a portion of the surge, the rapid proliferation of synthetic generation tools has enabled unprecedented levels of digital abuse and extortion. Organization leaders are calling on international governments to treat technology-facilitated exploitation as a public health emergency.

In response to the figures, safety organizations are urging governments and tech platforms to enact stricter safety guardrails on image generation systems and allocate more resources toward automated detection and immediate content removal.

Why it matters

  • Generative AI startups face escalating regulatory pressure and compliance demands to enforce robust safety guardrails on open and closed-source media tools.

  • Developers of computer vision and content detection models will see increased demand for enterprise safety systems from platforms mitigating legal liability.

  • Expect upcoming legislation targeting ’nudify’ software and synthetic media tools, increasing potential operational and legal risks for generative app creators.

Source: theguardian.com