NVIDIA's new surgical world model

 

2 weeks ago Nvidia Released Cosmos-H-Dreams for Real-Time Surgical-Robot Simulation

You can get all the details on HuggingFace.

• "Cosmos-H-Dreams takes an initial image and robot kinematics, then generates synthetic RGB video showing predicted future scene states. It does not generate or execute robot actions.
• Nvidia reports about 160 generated frames per second on one RTX PRO 6000, but that is a throughput figure rather than a disclosed end-to-end response-latency measurement.
• The code and 4.36-gigabyte checkpoint are available, although the repository’s promised dedicated technical report still points to a placeholder.
• CMR Surgical demonstrated a Versius integration in simulation. The capability is not part of the cleared Versius Plus system and is not intended for clinical decisions or patient care."

 "Starting from a single RGB frame and a live stream of robot kinematics, Cosmos-H-Dreams generates the evolving surgical scene in real time. A person can control it through a browser or Meta Quest, while a learned surgical policy can act inside the generated environment in closed loop.

These WFM environments, as they improve in quality, can become interactive R&D environments for closed-loop policy evaluation, synthetic data generation, failure-mode exploration, and surgical-skills research."

Blog: https://lnkd.in/gCiaXZwP
Model: https://lnkd.in/gprcCCQu

Code: https://lnkd.in/gXFphwfG 

Source: NVIDIA, LinkedIn