Unifying Physics-Based Humanoid Interaction with a Context-Conditioned Interaction Prior
Continuous behavior
Observe the scene and begin a single, continuous interaction.
Developing unified physics-based humanoid controllers that can navigate complex 3D scenes and manipulate objects remains a longstanding challenge. Existing approaches are often specialized for either locomotion or object-centric manipulation, or rely on task-specific reward engineering that does not scale well across diverse behaviors. We present CHIP, a unified, physics-grounded framework for learning reusable humanoid interaction skills from heterogeneous motion data. Central to our approach is a conditional interaction prior that models a context-dependent distribution over these skills within a shared discrete space. Our method is trained in three stages. We first learn physics-based motion-imitation policies that acquire grounded teacher behaviors from heterogeneous interaction data. We then distill these behaviors into a context-conditioned interaction prior that captures reusable motion structure across locomotion and manipulation. Finally, we initialize downstream task policies from the pretrained prior and adapt them through prior-regularized online RL post-training. Experiments on a diverse suite of humanoid interaction tasks show that our approach supports scene-aware locomotion, contact-rich object manipulation, and compositional behaviors such as environment-aware object transport and long-horizon skill sequencing, while producing smooth transitions and physically plausible motion.
A three-stage recipe turns heterogeneous demonstrations into a reusable interaction prior, then adapts it to new objectives without losing natural motion.
Specialist policies track retargeted human–scene and human–object motions inside physics simulation. Their rollouts provide grounded teacher behaviors spanning locomotion, whole-body object transport, and context-dependent interaction.
A VQ-VAE compresses teacher transitions into a shared 1,024-entry discrete codebook. A transformer then learns a categorical prior over skill tokens conditioned on humanoid history together with scene, object, and task context.
Each downstream task policy is initialized from the pretrained prior and optimized with PPO. KL regularization toward a frozen, CFG-guided prior encourages task adaptation while retaining interaction structure learned from the demonstrations.
Policies adapted from the same prior support scene navigation, object transport, and their composition in continuous episodes.
Rotate, step sideways, and reshape posture to traverse narrow gaps with fewer collisions.
Transport an object while continuously adapting whole-body motion to nearby scene geometry.
Compose navigation and manipulation in one continuous episode. The policy is learned through RL post-training with a scripted stage transition, without complete trajectories in the pretraining motion data.
The experiments test whether a shared interaction prior can support scene navigation, object relocation, and composition. Results follow the paper's in-distribution protocol, with an additional held-out-scene evaluation.
| Method | Scene navigation | Object relocation | Composition |
|---|---|---|---|
| CHIP Ours | 90.1 | 88.2 | 71.4 |
| TokenHSI | 93.3 | 84.8 | 66.8 |
| MaskedMimic | 33.8 | 44.1 | 0.0 |
Additional CHIP metricsNavigation collision score: 0.02 · Object final distance: 0.10 m
Values are reproduced from Table 1 in the paper. The comparison uses randomized in-distribution evaluation on scenes and object instances used for downstream training.
CHIP retains high navigation and composition success in held-out scenes.
If CHIP helps your research, please cite the paper.
@inproceedings{li2026chip,
author = {Li, Jianan and Chen, Xiao and Wong, Tien-Tsin},
title = {Unifying Physics-Based Humanoid Interaction with a Context-Conditioned Interaction Prior},
year = {2026},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
booktitle = {SIGGRAPH Asia 2026 Conference Papers},
series = {SA Conference Papers '26},
location = {Kuala Lumpur, Malaysia},
isbn = {979-8-4007-2842-6},
doi = {10.1145/3829340.3842357}
}