PDF Paper ArXiv arXiv GitHub Code Huggingface Dataset

Habitat-GS:

A High-Fidelity Navigation Simulator
with Dynamic Gaussian Splatting

ECCV 2026

Ziyuan Xia1 Jingyi Xu1 Chong Cui1 Yuanhong Yu1 Jiazhao Zhang2 Qingsong Yan3
Tao Ni4 Junbo Chen4 Xiaowei Zhou1 Hujun Bao1 Ruizhen Hu5 Sida Peng1,†


1. Zhejiang University 2. Peking University 3. XGRIDS 4. UDeer AI 5. Shenzhen University

Habitat-GS:

A High-Fidelity Navigation Simulator
with Dynamic Gaussian Splatting

arXiv preprint 2026

Ziyuan Xia1 Jingyi Xu1 Chong Cui1 Yuanhong Yu1 Jiazhao Zhang2 Qingsong Yan3
Tao Ni4 Junbo Chen4 Xiaowei Zhou1 Hujun Bao1 Ruizhen Hu5 Sida Peng1,†

TL; DR

We present Habitat-GS, a navigation-centric embodied AI simulator based on 3DGS and dynamic gaussian avatars. Compared to traditional mesh-based simulators, our simulation environment delivers photorealistic rendering. Habitat-GS is extended from facebook research's Habitat-Sim, and we integrated 3DGS scene rendering backend and a drivable gaussian avatar module into it. Our simulation platform is fully open-source and maintains complete compatibility with Habitat ecosystem. By introducing gaussian splatting to embodied simulators, we hope our work can facilitate future embodied AI research.


Comparison

Habitat-Sim (Mesh-based)
Habitat-GS (Ours)
Compared to traditional mesh-based simulators (left), our 3DGS-based simulator (right) preserves high-frequency visual details and view-dependent effects, and introduces realistic gaussian avatars, thus helping train more robust navigation agents. Click a video to toggle pause/play, and hover to zoom in.

Gallery of GS Scenes and Avatars

Habitat-GS supports scalable import of 3DGS assets. These include self-reconstructed GS scenes, public 3DGS datasets (e.g., InteriorGS), and generative pipelines like Marble.

More 3DGS Scene Demos in Habitat-GS

InteriorGS
Marble

Interactive Viewer

Explore 3DGS scenes in first-person perspective just as a navigation agent do in Habitat-GS!
Your action is restricted on Navigation Mesh, while 3DGS is responsible for photorealistic rendering.
★ indicates reflective scenes that highlight 3DGS-specific view-dependent effects.

🦞 HabitatAgent

HabitatAgent is an LLM-powered agent system built on top of Habitat-GS, enabling natural-language navigation,
MCP tool integration, and interactive scene exploration via a terminal chat interface.

Click the corner brackets to open the full stage breakdown.

Method

Method Pipeline

System overview of Habitat-GS. From left to right: Asset Preparation, where 3DGS scene assets and gaussian avatar assets are prepared respectively. Habitat-GS Simulation Environment, where the render engine performs 3DGS rasterization for scene gaussians and LBS deformation followed by rasterization for avatar gaussians, producing RGB-D observations. The NavMesh blocking module retrieves pre-computed proxy capsules at runtime and injects them into the NavMesh for obstacle blocking. Embodied Agent Loop, where the agent policy consumes sensor observations, executes actions, and receives rewards shaped by both navigation success and avatar collision.

Architecture

System Architecture

System architecture of Habitat-GS. Habitat-GS is built as a non-intrusive extension to Habitat-Sim. The system adopts a visual-navigation decoupling design principle, separating the visual rendering modules handled by the CUDA-based 3DGS rasterizer and LBS deformation, from the navigation module managed by the traditional NavMesh and injected proxy capsules. This allows for photorealistic agent observations without modifying the core Habitat-Sim navigation logic. For more details, please refer to our paper.

Bibtex

@inproceedings{xia2026habitat,
  title={Habitat-gs: A high-fidelity navigation simulator with dynamic gaussian splatting},
  author={Xia, Ziyuan and Xu, Jingyi and Cui, Chong and Yu, Yuanhong and Zhang, Jiazhao and Yan, Qingsong and Ni, Tao and Chen, Junbo and Zhou, Xiaowei and Bao, Hujun and others},
  booktitle={European Conference on Computer Vision},
  pages={306--323},
  year={2026},
  organization={Springer}
}