Vinces Yao Shunyu, chief AI scientist at Tencent, warned that AI systems will soon be capable of developing biological weapons, proposing a hybrid of open and closed models to manage safety and defense. Yao noted that while closed models implement strict safety filters, open-source models remain necessary for cybersecurity defense and detailed incident forensics, pointing to a past Hugging Face investigation where an open model was used after closed models refused the analysis.

The debate highlights tensions around safety guardrails, as techniques like ‘Abliteration’ allow users to easily strip refusal behaviors from open-weight models. In response, research and regulatory initiatives—including China’s TC260 standards body—are working on stronger pre-training safeguards and export controls to keep hazardous knowledge out of AI models entirely.

Why it matters

  • Safety filters on open-weight models remain easily bypassable, shifting industry focus toward removing hazardous knowledge during pre-training.

  • Enterprises must navigate evolving international security standards as regulators target biological and cyber risks in frontier models.

Source: the-decoder.com