The framework accepts two kinds of direction at once. Text prompts drive behaviors the team demonstrates, including a limp, a pick-and-put action, a stealthy walk, a victory dance, a lean-and-peek, and a zombie sit. Kinematic constraints handle spatial precision: root trajectories and waypoints, full-body keyframes, end-effector positions and rotations, or arbitrary combinations of those. The system also supports long-horizon goal reaching, where a constraint is specified beyond the current generation window, and real-time locomotion control through mouse waypoints and keyboard commands.
A hybrid representation and a two-stage denoiser
The method splits the motion into two representations. It combines explicit global root motion features with latent body embeddings, which the team frames as a way to balance trajectory control against generative efficiency. On top of that sits a two-stage autoregressive transformer denoiser that predicts the root motion first, then conditions the body-motion prediction on that root output.
Two additional pieces support the streaming behavior. A variable-length history context captures longer-term semantics to improve generation quality, and masked kinematic constraints allow spatiotemporally sparse conditioning that extends past the current window. The generative core follows the diffusion approach that has moved into character work elsewhere; we covered Motorica's raise to bring generative AI to character animation.
From on-screen characters to a physical humanoid
ARDY does not stop at rendered figures. The team integrated its output with a Unitree G1 humanoid robot through the SONIC physical tracking policy, using the generated motion to drive interactive robot control. That connection to hardware fits NVIDIA's broader robotics push; we covered the company's GTC keynote and its open humanoid robot dataset for training foundation models.
For production teams, the interactive framing separates ARDY from capture-first pipelines. Where markerless systems reconstruct motion from a performance, ARDY generates it from instructions; we covered Move AI's Gen 2 markerless capture as one point on that spectrum.
Published at SIGGRAPH with code and weights available
The paper appears in ACM Transactions on Graphics, Volume 45, Issue 4, Article 86, presented at SIGGRAPH, with DOI 10.1145/3811284. The authors are Kaifeng Zhao (NVIDIA, ETH Zürich), Mathis Petrovich (NVIDIA), Haotian Zhang (NVIDIA), Tingwu Wang (NVIDIA), Siyu Tang (ETH Zürich), and Davis Rempe (NVIDIA). The model weights are posted on Hugging Face alongside the code.
Because the code and weights are public, studios and robotics teams can test real-time, promptable motion generation against their own constraints instead of waiting for a product. The open release lets teams check how the models hold up on custom rigs and live control loops directly, rather than through the paper's demo clips.
