DPS-Net: Deep Polarimetric Stereo Depth Estimation

ICCV 2023

1State Key Lab of CAD & CG, Zhejiang University, 2Hong Kong University of Science and Technology
Pipeline overview


Stereo depth estimation usually struggles to deal with textureless scenes for both traditional and learning-based methods due to the inherent dependence on image correspondence matching. In this paper, we propose a novel neural network, i.e., DPS-Net, to exploit both the prior geometric knowledge and polarimetric information for depth estimation with two polarimetric stereo images. Specifically, we construct both RGB and polarization correlation volumes to fully leverage the multi-domain similarity between polarimetric stereo images. Since inherent ambiguities exist in the polarization images, we introduce the iso-depth cost explicitly into the network to solve these ambiguities. Moreover, we design a cascaded dual-GRU architecture to recurrently update the disparity and effectively fuse both the multi-domain correlation features and the iso-depth cost. Besides, we present new synthetic and real polarimetric stereo datasets for evaluation. Experimental results demonstrate that our method outperforms the state-of-the-art stereo depth estimation methods.

Cascaded Dual-GRU Architecture

Demonstration of Dual-GRU Architecture

We design a cascaded dual-GRU architecture to combine the iso-depth cost with the visual similarity. The cascaded dual-GRU architecture is composed of a regression GRU and an optimization GRU, which implement a regress-and-optimize recurrently updating paradigm. In the regression GRU, the disparity is regressed intuitively from the multi-domain correlation features of the actual disparity, which leverage the similarity of the RGB and polarization images to produce an initial prediction for the subsequent optimization. In the optimization GRU, the iso-depth cost is injected directly and introduced in an indirect manner by the virtual correlation feature. The optimization GRU fuses the geometric constraint and further rectifies the disparity.

Qualitative Results

Visualization of Qualitative Results Visualization of Qualitative Results



  title={DPS-Net: Deep Polarimetric Stereo Depth Estimation},
  author={Chaoran Tian, Weihong Pan, Zimo Wang, Mao Mao, Guofeng Zhang, Hujun Bao, Ping Tan, Zhaopeng Cui},
  booktitle={The IEEE/CVF International Conference on Computer Vision (ICCV)},