DeepSeek AI has released DeepSeek-V4.1-Flash, an open-weight Mixture-of-Experts multimodal model engineered to eliminate memory bottlenecks in long-horizon agent serving. The model features 552B backbone parameters, 196B Engram parameters, and a 1M-token context window while activating just 8B parameters during prefill and 16B during decode. Open weights have been released under an MIT license on Hugging Face alongside vLLM and SGLang support.
To achieve efficiency, DeepSeek introduced Cross-Layer Attention Reuse (CSA2) and Decoder SWA Bounded Replay, splitting the 40-layer backbone into a 20-layer causal encoder and a 20-layer decoder. This design allows prompt tokens to stop at the encoder, effectively halving prefill compute. Additionally, main KV cache storage is quantized to FP4 (E2M1), reducing the global KV cache footprint to 890 bytes per token—about 437 times smaller than DeepSeek-V1.
Pre-trained on 45 trillion multimodal tokens, the model matches DeepSeek-V4-Pro-Base on world knowledge and coding tasks using one-third of total parameters. Its serving optimizations ensure single-token decode FLOPs rise by only 25% when expanding context length from 4K to 1 million tokens.
Why it matters
FP4 quantization and cross-layer attention reuse dramatically drop serving costs and HBM requirements for high-throughput, 1M-context LLM deployments.
Open-weight MIT licensing enables developers to self-host massive agentic workflows previously restricted to costly, proprietary cloud APIs.
Architectural tricks like bounded replay and persistent DRAM pools eliminate SSD persistence needs, serving as a blueprint for next-gen inference infrastructure.
Source: marktechpost.com
