AI R&D lab Mirendil released Fern 0.1, its initial coding model built from scratch with a context window scaling up to 10 million tokens (equivalent to roughly 23 MB of code). Trained on PyTorch and JAX using public sources, the pretraining phase for its base model required under $10 million in compute, with AI agents generating over 99% of the code under human supervision.

Fern is designed specifically for technical R&D work rather than general conversational chat, enabling developers to load entire repositories, comprehensive log files, or multi-day agent traces simultaneously. Mirendil reported that the model’s loss curve continues to improve significantly beyond standard long-context limits.

Benchmark tests reveal mixed performance: Fern 0.1 outperforms models like Inkling and Nemotron 3 Ultra across several evaluations, but lags behind top-tier frontier systems on SWE-Bench Verified. The company noted that larger parameter variants (Mirendil-Base-Large and XL) are currently undergoing pretraining.

Why it matters

  • Enables processing of massive codebases and extensive agent trace logs within a single context window.

  • Demonstrates significant compute efficiency, reaching 10M context training for under $10 million.

  • Showcases agent-led software development, with over 99% of pretraining code authored by AI agents.

Source: marktechpost.com