GURecon: Learning Detailed 3D Geometric Uncertainties for Neural Surface Reconstruction

AAAI 2025

[Zesong Yang1, Ru Zhang1, Jiale Shi1]Co-Authors, Zixiang Ai1, Boming Zhao1,
Hujun Bao1, Luwei Yang2, Zhaopeng Cui†1,
1State Key Lab of CAD & CG, Zhejiang University, 2Simon Fraser University
†Corresponding Authors
Paper ArXiv Supplementary Code
>

Abstract

Neural surface representation has demonstrated remarkable success in the areas of novel view synthesis and 3D reconstruction. However, assessing the geometric quality of 3D reconstructions in the absence of ground truth mesh remains a significant challenge, due to its rendering-based optimization process and entangled learning of appearance and geometry with photometric losses. In this paper, we present a novel framework, i.e, GURecon, which establishes a geometric uncertainty field for the neural surface based on geometric consistency. Different from existing methods that rely on rendering-based measurement, GURecon models a continuous 3D uncertainty field for the reconstructed surface, and is learned by an online distillation approach without introducing real geometric information for supervision. Moreover, in order to mitigate the interference of illumination on geometric consistency, a decoupled field is learned and exploited to finetune the uncertainty field. Experiments on various datasets demonstrate the superiority of GURecon in modeling 3D geometric uncertainty, as well as its plug-and-play extension to various neural surface representations and improvement on downstream tasks such as incremental reconstruction.
pipeline

Fig. 1: System Overview. The proposed GURecon models a geometric uncertainty field supervised by the pseudo labels computed based on the multi-view geometry consistency. To deal with the view-dependent factors, additional decoupled fields are also learned and exploited to fine-tune the uncertainty field. With the predicted uncertainty fields, GURecon can boost the downstream tasks such as incremental reconstruction.

Comparison

Here, we present the reconstruction results and the corresponding geometric uncertainty predicted by Bayes' Rays and Ours.

GT Normal Pred Normal GT Unc Bayes' Rays Ours
We show the interactive reconstructed mesh and our estimated geometric uncertainty. Please wait for the model to load.

Application: Incremental Reconstruction

We apply our geometric uncertainty to the incremental reconstruction task and compare with current mainstream NeRF-based incremental reconstruction methods: ActiveNeRF (gaussian distributions) and UncertaintyNeRF (weight entropy). Uncertainty dynamically reflects the current quality of the reconstruction.
Uncertainty dynamically reflects the current quality of the reconstruction.
incre_compare

BibTeX

@inproceedings{yang2024gurecon,
    title     = {GURecon: Learning Detailed 3D Geometric Uncertainties for Neural Surface Reconstruction},
    author    = {Yang, Zesong and Zhang, Ru and Shi, Jiale and Ai, Zixiang and Zhao, Boming and Bao, Hujun and Yang, Luwei and Cui, Zhaopeng},
    booktitle = {AAAI},
    year      = {2025}
  }