David Finsterwalder turns Gaussian splat scans into sim-ready scenes for Isaac Sim and MuJoCo

XX / @DFinsterwalderShowcases

David Finsterwalder turns Gaussian splat scans into sim-ready scenes for Isaac Sim and MuJoCo
David Finsterwalder on X

David Finsterwalder shared on X a pipeline he is building that goes from a full environment scan captured as a Gaussian splat to a simulation-ready environment for NVIDIA Isaac Sim or Google DeepMind’s MuJoCo. He uses OpenAI’s GPT Astra models for the 3D modelling step and says moving from Astra 6 to 6.2 brought a clear improvement in 3D modelling and robot control.

Letting the model drive the robot directly also works, he writes, although it is slow because it pauses while thinking; he plans to use the recorded movements to fine-tune open-weight robot models such as pi 0.5, MolmoAct or the upcoming Gr00t 2. The pipeline is meant for workshops and for testing client use cases in simulation before larger investments in robot hardware. The post drew 52 likes and 49 bookmarks.