FreeTimeGS: Free Gaussian Primitives at Anytime Anywhere for Dynamic Scene Reconstruction

CVPR 2025


Yifan Wang1* Peishan Yang1* Zhen Xu1* Jiaming Sun1 Zhanhua Zhang2 Yong Chen2 Hujun Bao1 Sida Peng1 Xiaowei Zhou1x

1Zhejiang University    2Geely Automobile Research Institute
*Equal Contribution    xCorresponding Author

Real-Time rendering of dynamic 3D scenes with large and complex motions.

Abstract


This paper addresses the challenge of reconstructing dynamic 3D scenes with complex motions. Some recent works define 3D Gaussian primitives in the canonical space and use deformation fields to map canonical primitives to observation spaces, achieving real-time dynamic view synthesis. However, these methods often struggle to handle scenes with complex motions due to the difficulty of optimizing deformation fields. To overcome this problem, we propose FreeTimeGS, a novel 4D representation that allows Gaussian primitives to appear at arbitrary time and locations. In contrast to canonical Gaussian primitives, our representation possesses the strong flexibility, thus improving the ability to model dynamic 3D scenes. In addition, we endow each Gaussian primitive with an motion function, allowing it to move to neighboring regions over time, which reduces the temporal redundancy. Experiments results on several datasets show that the rendering quality of our method outperforms recent methods by a large margin. The code will be released for reproducibility.

Method


We represent a dynamic scene using Gaussian primitives that can appear anytime anywhere. Each Gaussian is assigned with a motion function to to model its movement. And its opacity is modulated by the temporal opacity function which control the impact of the Gaussian primitive over time. With this 4D representation, we further regularize the Gaussians with a 4D regularization loss and optimize the rasterization result with rendering loss for reconstructing a dynamic 3D scene from multi-view videos.

More Demos


More Real-Time Demos with slow motion playback.

More Street Reconstruction on the Waymo Dataset

Real-Time VR Demos


Real-Time VR Demos on Apple Vision Pro and Meta Quest 3

Baseline Comparisons


Comparisons with 4DGS, STGS and Deform-3DGS


Citation


@inproceedings{wang2025freetimegs,
  author  = {Wang, Yifan and Yang, Peishan and Xu, Zhen and Sun, Jiaming and Zhang, Zhanhua and Chen, Yong and Bao, Hujun and Peng, Sida and Zhou, Xiaowei},
  title   = {FreeTimeGS: Free Gaussian Primitives at Anytime Anywhere for Dynamic Scene Reconstruction},
  booktitle = {CVPR},
  year    = {2025},
  url     = {https://zju3dv.github.io/freetimegs}
}

Business Inquiries


For business inquiries and other collaboration opportunities, please fill out this form and we will get back to you soon.