BillBoard Splatting (BBSplat): Learnable Textured Primitives for Novel View Synthesis

David Svitov1,2
Pietro Morerio2
Lourdes Agapito3
Alessio Del Bue2
1Università degli Studi di Genova, Italy
2Istituto Italiano di Tecnologia (IIT) Genoa, Italy
3Department of Computer Science, University College London
Paper arXiv Video Code

Abstract



We present billboard Splatting (BBSplat) - a novel approach for 3D scene representation based on textured geometric primitives.

BBSplat represents the scene as a set of optimizable textured planar primitives with learnable RGB textures and alpha-maps to control their shape. BBSplat primitives can be used in any Gaussian Splatting pipeline as drop-in replacements for Gaussians. The proposed primitives close the rendering quality gap between 2D and 3D Gaussian Splatting (GS), enabling the accurate extraction of 3D mesh as in the 2DGS framework. Additionally, the explicit nature of planar primitives enables the use of the ray-tracing effects in rasterization.

Our novel regularization term encourages textures to have a sparser structure, enabling an efficient compression that leads to a reduction in the storage space of the model up to x17 times compared to 3DGS. Our experiments show the efficiency of BBSplat on standard datasets of real indoor and outdoor scenes such as Tanks&Temples, DTU, and Mip-NeRF-360. Namely, we achieve a state-of-the-art PSNR of 29.72 for DTU at Full HD resolution.

Method



Primitives Learnable Shape

3DGS

2DGS

BBSplat(Ours)


Results

Tanks&Temples

3DGS

Ours

3DGS-MCMC

Ours

2DGS

Ours


3DGS

Ours

3DGS-MCMC

Ours

2DGS

Ours

Mip-NeRF-360

3DGS

Ours

3DGS-MCMC

Ours

2DGS

Ours


3DGS

Ours

3DGS-MCMC

Ours

2DGS

Ours

DTU

3DGS

Ours

3DGS-MCMC

Ours

2DGS

Ours


3DGS

Ours

3DGS-MCMC

Ours

2DGS

Ours


Mesh Extraction

SuGaR

Ours

SuGaR

Ours


Ray-tracing Effects

BBSplat representation of the scene as explicit textured planes allows the implementation of ray-tracing effects during rasterization. The videos were obtained by rasterizing in Blender a set of planes with RGB and alpha textures. This way it is not necessary to spend extra time for extracting mesh representation and it allows to preserve photo realistic rendering while enabling rasterization effects and editing.


Concurrent Works

Method Learnable Shape RGB Texture Alpha Texture Base
3D Convex Splatting [1] ✅ ❌ ❌ 3DGS
Texture-GS [2] ❌ ✅ ❌ 3DGS
Textured Gaussians [3] ✅ ✅ ✅ 3DGS
SuperGaussians [4] ✅ ❌ ❌ 2DGS
HDGS [5] ❌ ✅ ❌ 2DGS
GSTex [6] ❌ ✅ ❌ 2DGS
Gaussian Billboards [7] ❌ ✅ ❌ 2DGS
BBSplat (Ours) ✅ ✅ ✅ 2DGS
  1. 3D Convex Splatting: Radiance Field Rendering with 3D Smooth Convexes
  2. Texture-GS: Disentangling the Geometry and Texture for 3D Gaussian Splatting Editing
  3. Textured Gaussians for Enhanced 3D Scene Appearance Modeling
  4. SuperGaussians: Enhancing Gaussian Splatting Using Primitives with Spatially Varying Colors
  5. HDGS: Textured 2D Gaussian Splatting for Enhanced Scene Rendering
  6. GStex: Per-Primitive Texturing of 2D Gaussian Splatting for Decoupled Appearance and Geometry Modeling
  7. Gaussian Billboards: Expressive 2D Gaussian Splatting with Textures


Inference speed & Storage Space



Textured 2D primitives allow to achieve improvement of quantitative metrics values and at the same time acceleration of inference (for 80K, 40K points) compared to 2D Gaussian primitives.





For the DTU dataset we provide better novel view synthesis quality than state-of-the-art compression methods while requiring less storage space.

BibTeX

@article{svitov2024billboard,
  title={BillBoard Splatting (BBSplat): Learnable Textured Primitives for Novel View Synthesis},
  author={Svitov, David and Morerio, Pietro and Agapito, Lourdes and Del Bue, Alessio},
  journal={arXiv preprint arXiv:2411.08508},
  year={2024}
}