Gaussian Splashing: Direct Volumetric Rendering Underwater A fast underwater 3D reconstruction method, that accounts for light attenuation and backscattering.

Nir Mualem Ben-Gurion University
Roy Amoyal Ben-Gurion University
Oren Freifeld Ben-Gurion University
Derya Akkaynak The Inter-University Institute for Marine Sciences and the University of Haifa
Paper Code (coming soon) TableDB dataset (coming soon)
Original Rendering (novel views) Normalized Backscatter Clean Rendering (novel views)
GaussianSplashing Rendered novel views of the Red Sea scene generated by the reconstructed scene model, alongside normalized backscatter images and clear medium-free images.


TableDB dataset: A newly added underwater dataset includes scene captures from different distances and viewpoints.

Abstract

Most useful features in underwater images are occluded by water. Consequently, underwater scenes present additional difficulties beyond the usual challenges of 3D reconstruction and rendering for in-air scenes. Naive applications of Neural Radiance Field methods (NeRFs) or Gaussian Splatting, while highly robust for in-air scenes, fail to perform underwater. Here we introduce Gaussian Splashing, a new method based on 3D Gaussian Splatting (3DGS), into which we incorporate an image formation model for scattering. Concretely, we introduce three additional learnable parameters to the rendering procedure, modify the depth estimation step for underwater conditions, alter the original loss term used in 3DGS, and introduce an additional loss term for backscatter. Our approach achieves state-of-the-art performance for underwater scenes and is highly efficient, with 3D reconstruction taking only a few minutes and rendering at 140 FPS, despite the complexities of underwater adaptation.

WaterSplatting overview
Method overview: Initially, we utilize Structure from Motion (SfM) to acquire an initial point cloud and camera poses. Subsequently, we commence the optimization process to refine the model based on our underwater rendering equation and modified tile rasterization, taking those distortions into account. We evaluate backscatter every 500 steps to ensure convergence towards the accurate medium coefficients using our approach.

Results

Gaussian Splashing 3DGS
Red Sea Scene A comparison of our method against 3DGS highlights the differences, showcasing the improvements in underwater scene reconstruction.
Gaussian Splashing 3DGS
Red Sea Scene: Notice how the improvement becomes significantly more pronounced as the view zooms out and the distance to the objects increases, with differences in far details clearly highlighted.
Rendered (154 FPS) Depth
Curacao Scene The Rendered view shows the scene’s colors and textures, while the Depth view represents object distances.
Rendered (163 FPS) Depth
Panama Scene rendered and depth.
WaterSplatting overview
Render and Training Time: A detailed comparison table showcasing the frames per second (FPS) performance during rendering, along with the training time (minutes) needed to build the model across various scenes.

Citation

Please use the following citation: 
@article{mualem2024gaussiansplashing,
  title={Gaussian Splashing: Direct Volumetric Rendering Underwater},
  author={Nir Mualem and Roy Amoyal and Oren Freifeld and Derya Akkaynak},
  journal={arXiv},
  year={2024}
}