Description
How do you turn a video of someone working into training data a robot can learn from? Join NVIDIA, RLWRLD, and Reactor to explore a pipeline that connects human demonstrations to robot actions, visual observations, and task instructions—and see how to run a Cosmos simulation checkpoint on Reactor.
Follow the process from synchronized video capture and hand–object reconstruction to robot-specific motion retargeting and rendering in NVIDIA Isaac Sim. Explore how RLWRLD uses NVIDIA Cosmos 3 features in its RLHND hand-tracking system, then examine research demos of world-model-generated observations for garment packing and different robot configurations.
Reactor will demonstrate how to run the Cosmos 3 Nano Sim Bimanual checkpoint on its platform for real-time world-model inference.
Built for robotics developers and researchers, the session covers practical data-quality choices, what the current results demonstrate, and what remains to be evaluated.
What You'll Learn:
- Follow the pipeline from synchronized human video to aligned robot actions, observations, and task instructions.
- Understand how RLHND uses NVIDIA Cosmos 3 features for hand pose, contact, and force estimation, and how reconstructed motion is retargeted to a robot.
- Compare Isaac Sim-rendered observations with experimental world-model-generated observations, including deformable objects and different robot configurations.
- How to run interactive world simulation models on Reactor
Have questions about human demonstration data, hand tracking, retargeting, or observation synthesis? Add them to the live chat for the NVIDIA + RLWRLD + Reactor Q&A.
Access more NVIDIA Cosmos developer resources and join our developer community:
📄Read the RLWRLD technical blog → https://www.rlwrld.ai/en/insight/blog/116
⬇️ Download Cosmos on Hugging Face → https://huggingface.co/collections/nvidia/cosmos3
📚 Explore Models & Datasets on GitHub → https://github.com/nvidia/Cosmos
👥 Join the Cosmos Community → https://discord.com/invite/nvidiaomniverse