GraftRay path-traces mesh objects into indoor Gaussian splat scenes

Jinxin Ai and Matthias Niessner present GraftRay, a method for physically consistent insertion of mesh objects into indoor scenes represented by 2D or 3D Gaussians. Existing approaches either harmonize appearance in image space, losing multi-view consistency, or approximate illumination with an environment map that cannot represent near-field area lights such as windows and lamps.
GraftRay recovers scene materials and surface normals with diffusion-based priors, models spatially extended emitters and solves their radiance in closed form, then jointly path-traces the scene Gaussians and inserted meshes. On FIPT it sets a new state of the art, outperforming recent baselines such as ComGS and EAG-PT in rendering fidelity and shadow accuracy.