Real-Time and Generalizable Physics-Based Animation via Self-Supervised Neural Skinning

Yuanhang Lei1,2, Baofei Liu1, Xingxuan Li1, Tao Cheng1, Zixuan Wang1, Boming Zhao1,
Yujun Shen3, Siyuan Huang2, Ruizhen Hu4, Peter Yichen Chen5, Hujun Bao1, Zhaopeng Cui1†

1State Key Laboratory of CAD&CG, Zhejiang University

2Beijing Institute for General Artificial Intelligence (BIGAI)

3Ant Group     4Shenzhen University     5University of British Columbia

† Corresponding author

arXiv Paper Code

TL;DR: PhysSkin++ learns generalizable, discretization-agnostic neural skinning fields directly from static 3D geometry through physics-informed self-supervision, then applies them in a feed-forward manner to meshes, point clouds, NeRFs, and Gaussian splats for real-time physics-based animation.

PhysSkin++ learns neural skinning fields from static meshes, point clouds, NeRFs, and Gaussian splats, enabling interactive control, multi-object interaction, and rig-complementary animation
Learned from unannotated static 3D geometries, PhysSkin++ enables interactive control, collision-aware multi-object interaction, and rig-complementary animation.

Abstract

Generalizing real-time physics-based animation across diverse 3D shapes and spatial discretizations remains challenging. In this paper, we introduce PhysSkin++, a physics-informed framework that learns continuous neural skinning fields from static geometries to construct compact deformation subspaces. Specifically, we propose a Neural Skinning Fields Autoencoder that predicts shape-dependent deformation subspaces without per-object network training. To learn these subspaces without simulation trajectories, we develop a Physics-Informed Self-Supervised Learning strategy that combines skinning-field normalization with conflict-aware gradient correction, stabilizing joint optimization under physical and orthogonality constraints. The learned representation supports efficient reduced dynamics with user control and contact interactions, together with direct animation binding to meshes, point clouds, and 3D Gaussians. For prescribed rig animation, we further introduce a self-supervised complementarity objective to learn complementary elastic responses without enforcing complementarity constraints at runtime. Experiments demonstrate cross-shape generalization of the learned skinning fields, real-time reduced simulation, and applications to controllable, collision-aware, and rig-complementary animation.

Method Overview

PhysSkin method pipeline

Given a static 3D shape, the framework samples volumetric cubature points for animation and surface points for shape encoding. A shape encoder produces shape latents, from which learnable queries extract handle latents through cross-attention. Spatial queries then attend to these handle latents and decode continuous neural skinning fields. Geometry-only, physics-informed self-supervision optimizes the fields under physical, smoothness, and orthogonality objectives.

Controllable and Collision-Aware Animation

PhysSkin++ supports diverse real-time physics-based animations through Dirichlet boundary conditions, interactive dragging, and energy-based boundary penalties for environmental contact.

Dirichlet Boundary Conditions

Prescribed regions remain fixed while the unconstrained domain deforms under external forces.

1 / 6

Interactive Control

Users manipulate localized regions of a deformable object through directly applied drag forces.

1 / 6

Single-Plane Environment Interaction

Boundary-penalty energies model contact between deformable objects and a single inclined plane.

1 / 12

Multi-Plane Environment Interaction

The same boundary-penalty formulation supports sequential contact across multiple planar boundaries.

1 / 6

Multi-Object Interaction

A point-based IPC-style barrier models inter-object contact, while boundary-penalty energies handle contact with the supporting floor.

Rigid–Deformable Interaction

Falling rigid bodies transfer collision responses to deformable objects through the shared contact formulation.

1 / 4

Deformable–Deformable Interaction

Multiple deformable objects are simulated jointly as inter-object contacts evolve over time.

1 / 6

Rig-Complementary Animation

PhysSkin++ enriches rig-based primary animation with physics-based complementary deformation.

Interactive Rig Control

Through an interactive rig interface, users manipulate bone poses while PhysSkin++ adds complementary elastic deformation.

1–2 / 10

User-Authored Rig Animation

PhysSkin++ enriches user-authored rig animation sequences with physics-based complementary elastic deformation.

1–2 / 10

Generalizable Self-Supervised Neural Skinning

PhysSkin++ predicts physically consistent neural skinning fields across different geometries.

Character Generalization

Category-General Objects

Comparison with Prior Methods

BibTeX

@inproceedings{lei2026physskin,
  title     = {PhysSkin: Real-Time and Generalizable Physics-Based Animation via Self-Supervised Neural Skinning},
  author    = {Lei, Yuanhang and Cheng, Tao and Li, Xingxuan and Zhao, Boming and Huang, Siyuan and Hu, Ruizhen and Chen, Peter Yichen and Bao, Hujun and Cui, Zhaopeng},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year      = {2026}
}