GENERATIVE AI

Apple Releases FastFrames Video Models for Interactive Generation, With Frames in as Little as 36 ms on One B200 GPU

A painted video frame of standing stones spills into a grassy 3D diorama, with trailing frames behind it and a 'FastFrames' nameplate below.

Apple released FastFrames on Hugging Face, two autoregressive video diffusion models that generate 480p video at interactive latency, with the 1.3B FastFrames-S producing a frame every 36 ms on one B200 GPU and the 14B FastFrames-L reaching 45 ms per frame on an eight-GPU B200 node. Apple says that speed enables real-time interaction, such as steering the camera with a keyboard or dragging objects with physics while the video generates.

  • Two sizes for two hardware setups. FastFrames-S runs at 36 ms per frame, or 28 responses per second, on a single NVIDIA B200 GPU. FastFrames-L runs at 45 ms per frame, or 22 responses per second, on an 8x B200 node, and at 158 ms on one GPU.
  • Apple says the speed comes without a quality hit. According to the FastFrames project page, FastFrames-S preserves the visual quality of Causal Forcing++ (CF++), which takes 272 ms per generation. Apple says FastFrames-L is comparable to Krea-Realtime in realism, prompt adherence, composition and text rendering while running over 10x faster; Krea-Realtime takes 1,675 ms on one GPU.
  • Over half the latency sat outside the diffusion step. The FastFrames paper finds that VAE decoding and KV cache updates account for over half of end-to-end inference latency in state-of-the-art models. The team replaced both with lightweight alternatives and sped up diffusion sampling with systems optimizations.
  • A faster LingBot ships alongside. Apple also released an optimized LingBot 1.3B that it says is 16x faster than the official model and allows fluid camera-conditioned control.
  • All FastFrames-L training checkpoints are public. FastFrames-L is distilled from Wan2.1-14B, and Apple released its causal teacher, intermediate distilled models and final networks, alongside the smaller FastFrames-S weights.

Lighter decoding, KV refinement and faster sampling bring a FastFrames-S frame to 36 ms

Autoregressive video diffusion models generate frames from new observations and inputs, which Apple's machine learning researchers describe as a promising medium for interactive visual experiences.

The paper says existing models are at least an order of magnitude too slow for fluid interaction. The team's profiling traced much of that delay to components usually treated as fixed implementation details.

FastFrames replaces VAE decoding and KV cache updates with lightweight alternatives, a pixel decoder and a KV refinement network, and adds systems optimizations to the two-step diffusion sampling. The repository lists these timings for one FastFrames-S profiling run at 480p on one GPU with a full KV cache:

  • Two DiT evaluations. 16.1 ms and 16.2 ms.
  • KV refinement. 2.2 ms.
  • Pixel decoder. 0.9 ms.
  • Total. 35.4 ms, against 36 ms reported in the paper.

The paper sets 50 ms or less as its interactive target. Spread across eight GPUs, FastFrames-S drops to 14 ms per generation, according to the project page.

Camera and physics controls respond while the video generates

The project page demos run a model's official implementation next to the same model with FastFrames optimizations, fed the exact same input at the same moment. One uses Robbyant's LingBot-World model with keyboard navigation; the other uses RealWonder with physics-based dragging.

Hands on a laptop keyboard beside a screen comparing two Stonehenge views: LingBot 1.3B Official at 832 ms versus a FastFrames version at 52 ms
FastFrames demo: LingBot camera control with and without FastFrames optimizations. Image: huggingface.co

We previously covered Skywork's Matrix-Game 3.0 world model, which targets real-time interactive scene generation at 720p.

For standard text-to-video generation, the included script outputs 832 × 480 video, 81 frames at 16 fps, from either FastFrames model.

FastFrames-L builds on Wan2.1-14B, with its full training chain released

Apple first distilled the Wan2.1-14B model into a two-step causal model, then trained the pixel decoder and KV refinement network on top of it.

That distillation follows the Causal Forcing++ recipe. FastFrames-S takes a different route: it keeps the two-step frame-wise Causal Forcing++ generator frozen and trains the networks around it.

The release includes four FastFrames-L checkpoints: a causal teacher trained on latents generated by Wan2.1-14B, the consistency-distilled model, the DMD-distilled model, and the final generator with its pixel decoder and KV refinement network. According to the project page, the 14B models produce more realistic lighting and shadows, more coherent compositions and more legible text than their 1.3B counterparts.

Running FastFrames requires Linux x86-64 with an NVIDIA GPU and a CUDA 12.8-compatible driver, per the GitHub repository. Licensing is split: LingBot-World v2 code and weights follow their upstream CC BY-NC-SA 4.0 license, while the rest of the FastFrames code falls under the Apple Sample Code License, so anyone building on the LingBot demo should check those terms first.

KEEP READING