Mundo · Customized World Model
We give Mundo the requests that were sent to Atlas. Each request holds a handful of photographs of a real place with their cameras and depth, together with the camera path of the output. Mundo reconstructs a 3D proxy of the place from the photographs and renders the same path with its video model, using the photographs as visual context.
Both videos follow the requested cameras frame by frame. Mundo renders the requested field of view at 640×480, and we show the band that matches the Atlas frame. Switch to the proxy to see the geometry that Mundo renders from.
A world model of a place should also let things happen in it. Mundo keeps the reconstructed proxy as an explicit scene program, so a simulator can move robots and objects through the same rooms, and the video model renders the result with the photographs as context. These cases use the same captures as above.
A Unitree G1 follows a route that a planner finds on the reconstructed floor, past the round table and toward the windows. Its gait comes from a locomotion policy running in MuJoCo, and Mundo renders the walk from a following camera and from the robot's head camera.
A quadruped robot follows the corridor and turns past the logo wall. The proxy carries its body and leg motion, and the low front camera shows what the robot itself would see along the same route.
The G1 walks up to the round table, takes the back of a swivel chair and steps back to pull it out. MuJoCo moves the chair on its casters, and the video model renders the grasp and the motion in the same office.
Every locker door is a spring-loaded hinge in the scene program. Released one after another, each door swings to its stop at about 100 degrees and settles after a small rebound. Two of the compartments hold a parcel, and all other doors stay shut.
We push four cartons off the shelves at different moments. MuJoCo resolves their falls and collisions, and the cartons come to rest in a pile in front of the shelf while the gaps they leave stay on the shelves.
Each chair at the round table receives a short push and a spin. The chairs roll on low-friction casters and turn around their columns, and the camera circles the table as they spread out.