DaFISH factors co-mounted flash lighting out of NeRF and Gaussian splat color

Yimeng Dou, Adam M. Saunders and colleagues present DaFISH, a plug-in for novel-view synthesis pipelines such as NeRF and Gaussian Splatting, aimed at captures where an active light is rigidly attached to the camera. As the camera moves, that co-mounted light changes the illumination direction at each surface, and neural renderers often bake the resulting appearance into surface color.
DaFISH factorizes per-point emitted color into albedo and a spherical-harmonic illumination factor, evaluates the SH illumination directly during training, and generalizes illumination to new camera poses without retraining. Averaged over held-out frames of five scenes, it attains the best mean PSNR (38.91 dB), SSIM (0.9613) and LPIPS (0.0582) among the compared methods. The work has been submitted to ICRA.