Nov 5, 2024

The Robot Report Listen to this article New Project GR00T workflows and AI world model tools are intended to help developers of robot dexterity, control, manipulation, and mobility. Source: NVIDIA NVIDIA Corp. today announced new artificial intelligence and simulation tools to accelerate development of robots including humanoids. Also at the Conference for Robotic Learning, Hugging Face Inc. and NVIDIA said they are combining their open-source AI and robotics efforts to accelerate research and development. The tools include the generally available NVIDIA Isaac Lab robot learning framework and six new robot learning workflows for the Project GR00T initiative to accelerate humanoid development. They also include new world-model development tools for video data curation and processing, including the NVIDIA Cosmos tokenizer and NVIDIA NeMo Curator for video processing. Hugging Face said its LeRobot open AI platform combined with NVIDIA AI , Omniverse and Isaac robotics technology will enable advances across industries including manufacturing , healthcare , and logistics . NVIDIA Isaac Lab to help train humanoids Isaac Lab is an open-source robot learning framework built on NVIDIA Omniverse , a platform for developing OpenUSD applications for industrial digitalization and physical AI simulation. Developers can use Isaac Lab to train policies at scale for all types of robot movement, from collaborative robots and quadrupeds to humanoids, said NVIDIA. A guide to migrating from Isaac Gym is available online, and NVIDIA Isaac Lab 1. is available now on GitHub. Project GR00T offers blueprints for general-purpose robots Announced at the Graphics Processing Unit Technology Conference (GTC) in March, Project GR00T aims to develop libraries, foundation models, and data pipelines to help the global developer ecosystem for humanoid robots. NVIDIA has added six new workflows coming soon to help robots perceive, move, and interact with people and their environments: GR00T-Gen for building generative AI-powered, OpenUSD-based 3D environments GR00T-Mimic for robot motion and trajectory generation GR00T-Dexterity for robot dexterous manipulation GR00T-Control for whole-body control GR00T-Perception for multimodal sensing “Humanoid robots are the next wave of embodied AI,” said Jim Fan, senior research manager of embodied AI at NVIDIA. “NVIDIA research and engineering teams are collaborating across the company and our developer ecosystem to build Project GR00T to help advance the progress and development of global humanoid robot developers.” Project GR00T now includes six new workflows to accelerate humanoid development. Source: NVIDIA Cosmos tokenizers minimize distortion As developers build world models, or AI representations of how objects and environments might respond to a robot’s actions, they need thousands of hours of real-world image or video data. NVIDIA said its Cosmos tokenizers provide high quality encoding and decoding to simplify the development of these world models with minimal distortion and temporal instability. The company said the open-source Cosmos tokenizer runs up to 12x faster than current tokenizers. It is available now on GitHub and Hugging Face . XPENG Robotics, Hillbot, and 1X Technologies are using the tokenizer. “NVIDIA Cosmos tokenizer achieves really high temporal and spatial compression of our data while still retaining visual fidelity,” said Eric Jang, vice president of AI at 1X Technologies, which has updated the 1X World Model dataset. “This allows us to train world models with long horizon video generation in an even more compute-efficient manner.”