YUBI-Real2Sim checks whether a 3DGS scene is good enough for VLA policy evaluation

Hiroki Nishizawa, Takehiko Ohkawa, Nicola Scianca, Tatsuya Matsushima, Shigeo Morishima and Kei Ota present YUBI-Real2Sim, a framework for 3DGS-based scene reconstruction and real-to-sim policy evaluation under dual fisheye wrist views. They note that policies relying on wrist-mounted fisheye cameras are heavily exposed to holes and floaters in wide-angle views, and that the reconstruction quality needed for reliable evaluation has been unquantified.
The pipeline combines 200-degree fisheye capture, 3DGS training, 3D foundation models for object modeling, and MuJoCo-based physical simulation, and reproduced the observed real-world policy ordering in eight of nine scene-task comparisons. A benchmark across two bimanual robot platforms, three scenes, three tasks and dual pi0.5-based policies measures rank agreement and policy sensitivity to counterfactual 3DGS artifacts and capture disturbances. The paper is under submission.