LF-GS: 3D Gaussian Splatting for View Synthesis of Multi-View Light Field Images

Abstract

3D Gaussian Splatting (3D-GS) has emerged as a groundbreaking approach for view synthesis. However, when applied to light field image synthesis, the issue of a too narrow field of view (FOV) that leaves some areas uncovered, compounded by the problem of data sparsity, significantly compromises the quality of synthesized views using 3D-GS. To overcome these limitations, we present LF-GS, a specialized 3D-GS variant optimized for light field image synthesis. Our methodology incorporates two key innovations. First, by harnessing the unique advantage of light field sub-aperture images that provide dense geometric cues, our method enables the effective incorporation of enhanced depth and normal priors derived from light field images. This allows for more accurate depth than monocular depth estimation. Second, unlike other methods that struggle to control the generation of unreasonable Gaussians, we introduce adaptive regularization mechanisms. These mechanisms strategically regulate Gaussian opacity and spatial scale during optimization, thereby preventing model overfitting and preserving essential scene details. Comprehensive experiments on our newly constructed light field dataset demonstrate that LF-GS achieves significant quality improvements over 3D-GS.