AI research lab MirroS has released Code-as-World, an open-source framework and agentic loop that reconstructs physical scenes from video footage into executable MuJoCo physics code. Grounded in the premise that video frames are evidence of physical scenes rather than their underlying mechanics, the system uses up to five rounds of abductive search to verify candidates against source video inputs. Open-sourced under the Apache 2.0 license, the model release includes 4B and 9B parameter checkpoints fine-tuned from Qwen3.5.
In benchmarks on the QuantiPhy validation set, the Code-as-World-VL-9B model scored a 55.4 MRA, surpassing Gemini-3.1 Flash (54.8) and exceeding previous open-weight baselines by roughly 15 points. The framework leverages vision models such as SAM 3 and VGGT-Omega to extract masks, tracks, and depth maps, compiling them into a structured scene file. Training incorporated supervised fine-tuning across 73,335 image-space QA pairs followed by Group Relative Policy Optimization (GRPO) on executable world VQA datasets.
By converting raw video into executable physics environments, the project creates synthetic training data complete with ground-truth physical labels like mass, velocity, and contact forces. This approach addresses key limitations in video prediction models, providing structured simulation environments for downstream AI and robotics tasks.
Why it matters
Provides robotics and world-model developers an automated pipeline to extract ground-truth physics and simulation assets directly from real-world video footage.
Demonstrates that combining agentic verification loops with execution engines (MuJoCo) outperforms static video generation for physical reasoning tasks.
Delivers open-source, Apache 2.0-licensed models (4B and 9B) that beat proprietary frontier baselines on physical QA benchmarks.
Source: marktechpost.com
