Coded-aperture cameras are designed end-to-end against 3DGS scene representations

arXivarXivPapers

Coded-aperture cameras are designed end-to-end against 3DGS scene representations
arXiv / Chengyang Yan and Donald G. Dansereau

Chengyang Yan and Donald G. Dansereau present a differentiable framework for jointly designing complex camera systems and downstream perception tasks. They use it to design two coded-aperture depth-from-defocus cameras, monocular and stereo, by jointly optimizing the coded-aperture mask, camera parameters, and a depth-estimation network end-to-end.

Scenes are represented with 3D Gaussian Splatting reconstructed from real-world captures, providing a differentiable image-formation process for gradient-based optimization over design spaces the authors say were intractable for prior differentiable approaches. They report that the method improves over existing coded-aperture design and non-differentiable camera-design baselines.