Learning Spherical Radiance Field for Efficient 360° Unbounded Novel View Synthesis IEEE Transactions on Image Processing 2024
- Minglin Chen Sun Yat-sen University
- Longguang Wang Sun Yat-sen University
- Yinjie Lei Sichuan University
- Zilong Dong Alibaba Group
- Yulan Guo Sun Yat-sen University
Abstract
Novel view synthesis aims at rendering any posed images from sparse observations of the scene. Recently, neural radiance fields (NeRF) have demonstrated their effectiveness in synthesizing novel views of a bounded scene. However, most existing methods cannot be directly extended to 360° unbounded scenes where the camera orientations and scene depths are unconstrained with large variations. In this paper, we present a spherical radiance field (SRF) for efficient novel view synthesis in 360° unbounded scenes. Specifically, we represent a 3D scene as multiple concentric spheres with different radii. In particular, each sphere encodes its corresponding layered scene into implicit representations and is parameterized with an equirectangular projection image. A shallow multi-layer perceptron (MLP) is then used to infer the density and color from these sphere representations for volume rendering. Moreover, an occupancy grid is introduced to cache the density field and guide the ray sampling, which accelerates the training and rendering procedures by reducing the number of samples along the ray. Experiments show that our method can well fit 360° unbounded scenes and produces state-of-the-art results on three benchmark datasets with less than 30 minutes of training time on a 3090 GPU, surpassing Mip-NeRF 360 with a 400x speedup. In addition, our method achieves competitive performance in terms of both accuracy and efficiency on a bounded dataset.
Methodology
Illustration of our method for 360° unbounded novel view synthesis. (a) a 3D scene is represented using multiple concentric spheres with different radii, where each sphere is parameterized by an equirectangular projection (ERP) image. ERP images are stacked as an ERP volume. (b) Multi-resolution ERP volumes with hashing encoding are used to maintain the trainable features as input to the following shallow MLPs. (c) The density σ and color c predicted from the MLPs are used in the volumetric rendering algorithm to synthesize an image.