GeoGaussian: Geometry-aware Gaussian Splatting for Scene Rendering

↣ Rendering of views that have already been seen

↣ Rendering of views that are novel and challenging

Abstract

During the Gaussian Splatting optimization process, the scene's geometry can gradually deteriorate if its structure is not deliberately preserved, especially in non-textured regions such as walls, ceilings, and furniture surfaces. This degradation significantly affects the rendering quality of novel views that deviate significantly from the viewpoints in the training data.

To mitigate this issue, we propose a novel approach called GeoGaussian. Based on the smoothly connected areas observed from point clouds, this method introduces a novel pipeline to initialize thin Gaussians aligned with the surfaces, where the characteristic can be transferred to new generations through a carefully designed densification strategy. Finally, the pipeline ensures that the scene's geometry and texture are maintained through constrained optimization processes with explicit geometry constraints. Benefiting from the proposed architecture, the generative ability of 3D Gaussians is enhanced, especially in structured regions. Our proposed pipeline achieves state-of-the-art performance in novel view synthesis and geometric reconstruction, as evaluated qualitatively and quantitatively on public datasets.

BibTeX

@inproceedings{liopenstructure,
  author = {Li, Yanyan and Lyu, Chenyu and Di, Yan and Zhai, Guangyao and Lee, Gim Hee and Tombari, Federico},
  title = {GeoGaussian: Geometry-aware Gaussian Splatting for Scene Rendering},
  year = {2024},
  booktitle = {arXiv}}

Reference

  [3DGS]Kerbl, B., Kopanas, G., Leimkühler, T., & Drettakis, G. (2023). 
          3D gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4), 1-14.
  [LightGS]Fan, Z., Wang, K., Wen, K., Zhu, Z., Xu, D. and Wang, Z., (2023). 
          Lightgaussian: Unbounded 3d gaussian compression with 15x reduction and 200+ fps. arXiv preprint arXiv:2311.17245.