Nvidia has published research details on GR00T 2, an AI foundation model designed to train robotic arms without requiring extensive manual teleoperation. Operating as a world-action model, GR00T 2 simultaneously predicts future control actions and generates matching video frame sequences of the robot performing the task. This architecture allows the system to transfer human movements recorded on video into control commands for different robot body designs.

To build the foundation model, Nvidia trained GR00T 2 on a cluster of over 1,000 GB300 chips using 400,000 hours of head-mounted video depicting human manual labor. During task fine-tuning, operators demonstrate steps using wearable tracking gear. The model converts these human demonstrations by projecting the robot’s limbs into the predicted video feed. In physical trials, robots trained this way successfully completed multi-step tasks like cutting fruit, lighting matches, and sorting packages.

For complex assembly sequences, Nvidia combined GR00T 2 with high-level language models such as OpenAI’s GPT-6 Astra. The language model breaks long procedures into sub-tasks and runs error-correction routines, raising successful multi-part toy assembly from 23 percent to over 97 percent. Nvidia also developed a lightweight variant, GR00T 2 Turbo, optimized to run directly on edge devices like the Jetson Thor onboard computer for mobile and bipedal robots.

Why it matters

  • Reduces teleoperation overhead by converting video of human actions directly into robot control policies.

  • Combines vision-action models with high-level LLMs to enable complex, error-correcting task assembly.

  • Enables low-latency execution of foundation robotics models on edge hardware like Jetson Thor.

Source: the-decoder.com