Aleph Alpha has released Kolibri, an open-weight Mixture-of-Experts (MoE) model tailored for German and English enterprise and public sector use. Built with 78.1 billion total parameters, the architecture activates just 3.46 billion parameters per token and supports context windows up to 1,048,576 tokens. The model is released under an Apache 2.0 license on Hugging Face, with FP8 checkpoints runnable on single B200, B300, or H200 GPUs served via vLLM.
Developed end-to-end in Germany and Finland, the pipeline was explicitly built to comply with the EU AI Act, GDPR, and the EU General-Purpose AI Code of Practice by redacting personal data prior to training. Pre-training spanned 20 trillion tokens, followed by mid-training and long-context stages, alongside post-training using reinforcement learning and supervised fine-tuning. Evaluation results show Kolibri leading benchmarks such as GPQA Diamond (84.3) and AIME 2025 (96.9) while achieving faster decoding speeds than previous iterations.
Why it matters
Offers European enterprises an EU AI Act and GDPR-compliant open-weight MoE model optimized for bilingual German-English workloads.
Enables cost-efficient, low-latency deployments by activating only 3.46B parameters per token, fitting FP8 checkpoints onto single high-end GPUs.
Provides dynamic context length scaling via sliding-window attention, supporting long context windows with significantly reduced memory growth.
Source: marktechpost.com



