InfoNeRF: Ray Entropy Minimization
for Few-Shot Neural Volume Rendering
Mijeong Kim
Seonguk Seo
Bohyung Han
Seoul National University
arXiv 2021
[Paper]
[Video]
[Github]

NeRF

NeRF (depth)

InfoNeRF

InfoNerF (depth)

InfoNeRF(ours) achieves outstanding quality of rendered images with 4 input views having wide baseline
and does not require any other external data/data structures or additional learnable parameters.

Abstract

We present an information-theoretic regularization technique for few-shot novel view synthesis based on neural implicit representation. The proposed approach minimizes potential reconstruction inconsistency that happens due to insufficient viewpoints by imposing the entropy constraint of the density in each ray. In addition, to alleviate the potential degenerate issue when all training images are acquired from almost redundant viewpoints, we further incorporate the spatially smoothness constraint into the estimated images by restricting information gains from a pair of rays with slightly different viewpoints. The main idea of our algorithm is to make reconstructed scenes compact along individual rays and consistent across rays in the neighborhood. The proposed regularizers can be plugged into most of existing neural volume rendering techniques based on NeRF in a straightforward way. Despite its simplicity, we achieve consistently improved performance compared to existing neural view synthesis methods by large margins on multiple standard benchmarks.

Overview Video

Coming Soon!



Results

Novel view synthesis on the Realistic Synthetic 360 Dataset

Experiments in 4-view setting


Qualitative results with 4 input views are shown on the top of the page. Please check out here for more results.

Experiments in 2-view setting

NeRF
DietNeRF
InfoNeRF


Novel view synthesis on the ZJU-MoCap Dataset (4-view)

Ground-truth
NeRF
NV
InfoNeRF

Novel view synthesis on the DTU MVS Dataset (3-view)

Ground-truth
NeRF
InfoNeRF w/o L_kl
InfoNeRF


Bibtex

 
        @article{kim2021infonerf},
            title = {InfoNeRF: Ray Entropy Minimization for Few-Shot Neural Volume Rendering},
            author = {Mijeong Kim and Seonguk Seo and Bohyung Han},
            journal = {arXiv.org}
            year = {2021},
        }