AOV-GS steers 3DGS exploration toward semantically novel views, using frozen SAM and CLIP

Rodion Shkokov, Tatiana Zemskova and Dmitry Yudin present AOV-GS, an active 3D Gaussian mapping system that guides exploration with frozen foundation models, published at AIST 2026. During coarse exploration it re-ranks geometrically promising candidate views by CLIP mask-embedding novelty computed from SAM instance segmentation, biasing the trajectory toward semantically informative regions, and a lightweight scene-specific language field is later fitted offline to the collected CLIP embeddings rather than to manually annotated masks.
In a controlled comparison that uses the same SAM-CLIP stack, AOV-GS improves average language-field mIoU over geometry-only active planning by 1.3 percentage points on Replica (34.6% on eight scenes) and by 1.8 points on Matterport3D (23.1% on five scenes). Novel-view PSNR stays comparable to the geometry-only baseline.