PrismML released Ternary Bonsai 2 27B, an open-weights (Apache 2.0) ternary-weight version of Qwen3.8 27B. The model reduces memory requirements from 53.80 GB in FP16 to 5.93 GB while retaining 98.2% of its parent model’s performance across 20 benchmarks, according to the company. Featuring 27.36 billion parameters, it accepts text and images, supports a 262K context window, and runs on consumer hardware including 16 GB laptops and local GPUs via PrismML’s llama.cpp fork or MLX runtime.
The compression relies on a 1.72 bit-per-weight scheme using ternary values (-1, 0, +1) alongside blockwise Hadamard rotations. While broad benchmark performance remains close to the baseline, PrismML reported higher degradation on long-horizon agent tasks, where performance dropped to roughly 75% of the original model on SWE-bench Verified and Terminal-Bench 2.1.
All metrics were reported by PrismML from tests on its custom kernels and have not been independently verified. On hardware performance, the company claims an RTX 5090 reaches 142.5 tokens per second, while Apple’s M5 Max chip hits 46.8 tokens per second using its custom packings.
Why it matters
Enables local deployment of 27B-class multimodal models on consumer-grade hardware like 16 GB laptops and single GPUs.
Dramatically reduces edge AI compute costs, claiming 40% higher energy efficiency than full-precision 8B models.
Highlights persistent trade-offs in extreme quantization, as agentic long-horizon reasoning degrades faster than standard benchmark accuracy.
Source: marktechpost.com
