Researchers from Stanford and Caltech have developed HomeBody, a system that connects OpenAI’s GPT-6 Astra directly to a Unitree G1 humanoid robot to perform household tasks. The setup allowed the robot to autonomously navigate an unfamiliar kitchen, tidy up, and retrieve items from drawers without relying on traditional pre-trained intermediate control layers.

Instead of specialized control pipelines, HomeBody utilizes Astra as a swappable vision-language model that directly calls an extensible skill library for navigation and manipulation. The robot maps unfamiliar spaces by generating a digital twin in Nvidia’s Isaac Sim and maintaining a spatial memory log, allowing it to track objects even after they leave its direct field of view.

Despite successful execution and self-correction capabilities, the researchers noted key technical bottlenecks, including inference latency from the vision-language model, overheating finger servos, and high compute requirements. The project’s code has been made publicly available on GitHub as OpenAI prepares to re-enter the robotics market.

Why it matters

  • Direct VLM-to-skill control bypasses traditional robotics pipelines, accelerating physical AI deployment in unmapped environments.

  • Hardware limitations such as servo thermal constraints and VLM latency remain primary bottlenecks for real-time robotic deployment.

  • OpenAI’s underlying model integration signals growing convergence between frontier vision-language models and embodied robotics.

Source: the-decoder.com