3D Gaussian Splatting for Real-Time Radiance Field Rendering
Radiance field methods, starting with NeRF, have achieved high-quality novel-view synthesis from a set of photos. However, volumetric MLPs are slow to train and render. We propose an explicit representation: a set of 3D Gaussians that can be projected and composited via a tile-based GPU rasterizer.
We represent the scene as a set of N 3D Gaussians, each with a mean μ, a covariance matrix Σ, an opacity α, and spherical-harmonic (SH) color coefficients for view-dependent appearance:
For optimization stability Σ is parameterized via a rotation R (quaternion) and a diagonal scale S:
We project each 3D Gaussian to 2D, sort by depth inside 16×16 tiles, and composite front-to-back using α-blending identical to NeRF's volume rendering:
The rasterizer is fully differentiable and written in CUDA; gradients flow through the α-blending to position, scale, rotation, opacity, and SH coefficients.
We supervise rendered images with the photometric L1 loss combined with a D-SSIM term:
We use Adam with exponentially-decaying learning rates on positions (1.6×10⁻⁴ → 1.6×10⁻⁶). Opacities, scales, rotations and SH coefficients have separate learning rates. Training runs 30,000 iterations.
Every 100 iterations, we densify and prune Gaussians based on the magnitude of the view-space position gradient ∇pL:
- Clone: small Gaussians (‖s‖ < τs) with high gradient → duplicate and nudge along gradient.
- Split: large Gaussians with high gradient → split into two smaller Gaussians with
s ← s/1.6. - Prune: Gaussians with opacity α < εα or world-space scale > 10% of scene extent are removed.
We implement the rasterizer in CUDA and the optimizer in PyTorch. MipNeRF360 outdoor scenes train to 27+ PSNR in ~30 minutes on an A6000 and render at 130 FPS at 1080p.
Hyperparameters: SH degree 3, densification from step 500 to 15000, opacity reset every 3000, gradient threshold 2×10⁻⁴, λssim = 0.2.
view 0 · front21.8
view 1 · side22.6
view 2 · back22.1
view 3 · wide23.0