World2HOSI orchestrates experts inside 3DGS scenes to generate long-horizon interactions

Jiushu Li, Haoran Wang, Yumeng Liu, Zhiyang Dou, Yuan Liu, Yuexin Ma and Cheng Lin present World2HOSI, a multi-expert orchestration harness that generates continuous human-object-scene interaction data from 3D Gaussian Splatting scenes and natural-language instructions. A persistent scene state is built from decoupled background and object assets; a harness plans action stages, invokes existing HOI and HSI experts through their native interfaces, and writes each action’s effects back into the world.
Bounded residual optimization and contact-driven physics simulation are used to reduce physical artifacts while preserving interaction intent. Experiments and a perceptual user study show that World2HOSI achieves the best task completion and interaction quality on the evaluated HOSI tasks, while generating more diverse data.