GEMS stores a persistent 3DGS map that updates only where the scene has changed

arXivarXivPapers

GEMS stores a persistent 3DGS map that updates only where the scene has changed
arXiv / Ling-Chia Chen, Abdelrhman Werby and colleagues

Ling-Chia Chen, Abdelrhman Werby, Kaan Aytekin and Kai O. Arras present GEMS, a framework that combines a persistent spatio-temporal 3D Gaussian Splatting map with an agentic reasoning pipeline. The map is reconstructed once and then updated only in regions where the scene changes, storing those modifications while preserving visual detail across time, and the agent can render novel views on demand to answer spatio-temporal questions and ground objects from natural-language queries.

The authors also introduce GEMS-Bench, with 3 real-world and 12 synthetic environments and 473 question-answer pairs. GEMS outperforms recent baselines in 3D object grounding and spatiotemporal question answering, while achieving rendering quality on par with dedicated scene-update methods.