NVIDIA releases Cosmos3-H-Dreams, an open real-time surgical world model trained on clinical gallbladder surgery that runs at 30 fps on one GPU

Why it matters

【Key points】

NVIDIA released Cosmos3-H-Dreams on Hugging Face on Oct 20, a real-time surgical world model. Given one endoscopic frame and commands for two robot arms, it generates the next stretch of surgical video, with tissue deforming as instruments grasp and pull. NVIDIA calls it the first real-time generative surgical simulator trained on clinical cholecystectomy recordings with live kinematic control.

1. Data and base: fine-tuned on clinical gallbladder-removal recordings from CMR Surgical's Versius robot (64 recordings, ~526 unique minutes per the model card) on Cosmos3-H-Embodied-Nano, built on NVIDIA Cosmos 3. About 15.2B parameters, 832x480 output.

2. Real time: a bidirectional teacher is distilled into a block-by-block causal student; 4-step and 2-step checkpoints run on NVIDIA's FlashDreams streaming library.

3. Speed (NVIDIA's tests): 30 fps (4 steps) and 43.2 fps (2 steps) on one RTX PRO 6000 Blackwell Workstation GPU.

4. Uses: testing surgical robot policies, teleoperated exploration and synthetic data. NVIDIA says it is a research simulator, not validated for patient care.

5. License: NVIDIA Open Model License, commercial use allowed; code, weights and training recipe are public.

【Verifiable】

NVIDIA blog: https://huggingface.co/blog/nvidia/cosmos3-h-dreams

Weights: https://huggingface.co/nvidia/Cosmos3-H-Dreams

Base model: https://huggingface.co/nvidia/Cosmos3-H-Embodied-Nano

Code: https://github.com/isaac-for-healthcare/Cosmos3-H-Dreams

FlashDreams: https://github.com/NVIDIA/flashdreams

【Our read】

1. Surgical robots cannot learn by trial and error on patients; a simulator that responds to commands live lets policies be tested at scale first.

2. It covers one procedure, two instruments and one vendor's robot, trained on a few dozen recordings; generalization still needs outside teams and surgeons to judge.

3. All speed and quality figures are NVIDIA's own.