Chinese AI labs Z.ai and Alibaba’s Qwen team independently released open-weight mixture-of-experts (MoE) models featuring nearly identical architectural configurations. Z.ai launched GLM-5.3-Flash, a 320-billion parameter multimodal model with 18 billion active parameters under an MIT license. Simultaneously, Alibaba unveiled Qwen3.8-Flash-Next, a 125-billion parameter model with 6 billion active parameters serving as an architectural preview for the Qwen4 family.
Despite being developed separately, both architectures converged on a 3:1 ratio of linear attention to full attention layers, a compressed indexer strategy capped at 2,048 tokens for context selection, and four gated residual stream branches. Both engineering teams also utilized the Muon optimizer during training. GLM-5.3-Flash offers a 1-million token context window trained on a 30-trillion token multimodal dataset, while Qwen3.8-Flash-Next reduced required training compute to one-ninth that of its predecessor.
The alignment between these releases signals an industry convergence toward hybrid linear-attention MoE architectures designed to optimize inference cost and memory footprints. By offloading long-context processing to constant-memory linear layers while preserving full attention for targeted retrieval, both models achieve long-context functionality at reduced compute costs.
Why it matters
Signals structural convergence in frontier MoE architecture, favoring 3:1 hybrid linear-attention designs to lower inference costs.
Provides enterprise buyers with open-weight options supporting up to 1M context windows at lower price points.
Demonstrates significant training efficiency gains, with Alibaba reporting architecture changes cut compute requirements by nearly 90%.
Source: marktechpost.com
