DynOLAT

DynOLAT OLAT Video Prediction for
Controllable Dynamic Full-Body Relighting

SIGGRAPH Asia 2026 TOG

1 Zhejiang University 2 Cornell University 3 4DV.ai 4 Ant Group 5 Zhejiang Lab

* Equal contribution. † Corresponding author.

arXiv Paper Code not released yet Dataset Video

One video. Explicit light directions. Controllable relighting.

Demo video

Download video
Full supplementary video · 4:48 · 720p

Controlling Light, Preserving Motion

Insights

DynOLAT relights full-body human performances from monocular flat-lit video. For videos captured under uncontrolled illumination, a lightweight delighting model first maps the input toward the flat-lit training domain. The OLAT predictor then generates temporally consistent One-Light-At-a-Time (OLAT) basis videos, each showing the same performance illuminated from a single direction. By linearly combining these basis videos with adjustable weights, DynOLAT controls lighting direction and relative intensity. To train this model, we build a dynamic capture pipeline and collect over 24 million frames from 224 identity-outfit combinations. Experiments with the trained model show that DynOLAT responds more faithfully to variations in target lighting direction and relative intensity than the tested baselines, enabling controllable video relighting while preserving fine appearance details and temporal coherence.

Method

Given a flat-lit video and a target light direction, our model predicts the corresponding OLAT basis video. We then combine predictions for different directions in linear color space to synthesize the performance under a target environment map.

DynOLAT pipeline: a flat-lit input video and spherical harmonic light embeddings condition the Bridge Matching Video Model to predict OLAT videos, which are combined in linear color space to produce relit videos under new environment maps.
Light-conditioned video prediction produces temporally coherent OLAT bases. Combining these bases in linear color space yields relit videos under new environment maps.

  1. OLAT Video Prediction

    Spherical harmonic embeddings encode target light directions. The video model predicts a corresponding sequence of OLAT basis frames for each direction.

  2. Latent Bridge Matching

    Bridge matching transports flat-lit source latents toward OLAT target latents, reducing distribution mismatch while preserving temporal structure and identity details.

  3. Linear Space Combination

    We combine the predicted OLAT videos in linear color space using illumination weights to synthesize the performance under a target environment map.

Dataset

HF Dataset

24M+captured frames

224identity-outfit combinations

4Kmulti-view capture

Capture Setup

Our LightStage has 101 addressable light units. The dynamic and static subsets use 68 and 70 synchronized cameras, respectively.

Coverage

189 dynamic and 35 static identity-outfit combinations, spanning varied clothing, motion, and viewpoints. Different outfits of one person count separately.

Paired supervision

Time-multiplexed flat-lit and OLAT frames are aligned using optical flow to form paired supervision for learning light responses.

More Results

Scope & limitations

The system uses delighting to adapt videos captured under uncontrolled lighting to the flat-lit domain used to train the OLAT predictor. However, strong cast shadows and colored illumination can persist after delighting. The pretrained video VAE limits the reconstruction of fine details, and generating and storing many OLAT basis videos adds computational and storage costs. Consistency across OLAT predictions and camera views is not guaranteed. See the supplementary analysis for failure cases.

Acknowledgments

We thank Dongli Tan for LightStage camera calibration and Linxian Wu for providing paired outdoor environment panoramas and reference images captured under the same lighting conditions.

We would also like to thank Jiaming Sun, Zhiyuan Yu, Yunzhi Yan, Shangzhan Zhang, Li Ma, and Sida Peng for their insightful discussions.

Citation

@article{liu2026dynolat,
  author = {Liu, Taoming and ShenTu, Yinji and He, Guangzhao and Jin, Yudong and Ye, Yukang and Zhang, Siyu and Jia, Fan and Hua, Wei and Bao, Hujun and Zhou, Xiaowei},
  title = {{DynOLAT}: {OLAT} Video Prediction for Controllable Dynamic Full-Body Relighting},
  journal = {ACM Transactions on Graphics},
  number = {6},
  volume = {45},
  month = {December},
  year = {2026}
}