Apple FastFrames Release: 36ms Video, How It Works
Apple released FastFrames on Hugging Face, a family of autoregressive two-step video diffusion models that generate 480p video at interactive latency. The small model, FastFrames-S (1.3B), runs at 36 ms per frame, or 28 responses per second, on a single NVIDIA B200 GPU. The large model, FastFrames-L (14B), runs at 45 ms per frame, or 22 responses per second, on an 8x B200 node. The paper, arXiv:2611.23226, was submitted on November 17, 2026 by Gene Chou, Guandao Yang, Fujun Luan, Lars Mescheder, Wei Dong, Stephan Richter and Vladlen Koltun.
The headline number is not the most interesting part. The authors profiled the inference pipeline of state-of-the-art models and found that operations usually treated as fixed implementation details, VAE decoding and KV cache updates, account for over half of end-to-end inference latency. Most speed work in video generation targets the denoising loop. This result says the loop was never the whole problem, and that the plumbing around it can cost more than the generation itself. If that profiling holds up on other stacks, a lot of published latency comparisons are measuring the wrong bottleneck.
The Profiling Finding Behind the Speed
FastFrames challenges the assumption that the heavy decoder and cache machinery are indispensable, and replaces them with lightweight alternatives. The model card publishes a component breakdown for FastFrames-S at 480p with a full KV cache. The two DiT evaluations cost 16.1 ms and 16.2 ms. The replacement parts are far cheaper: KV refinement takes 2.2 ms and the pixel decoder takes 0.9 ms, for a measured total of 35.4 ms, which the paper rounds to 36 ms.
Read that table carefully and the design becomes clear. After the replacements, almost all remaining latency is the generator itself, split across its two steps. The fixed costs that used to dominate have been compressed to about 3 ms combined. That is why the authors can claim that FastFrames-S matches the generation quality of a state-of-the-art model that is nearly 8x slower: the comparison model is spending most of its time outside the denoising steps that quality actually depends on.
How the Two Models Are Built
Both models are autoregressive, which means each new frame is generated from new observations and inputs. That is the property that makes the family suited to interactive uses such as camera-conditioned and physics-conditioned video generation, where the next frame has to respond to what the user just did. A clip generator that renders a finished file offline does not need this property. An interactive one cannot work without it, and it cannot feel interactive if each response takes hundreds of milliseconds.
FastFrames-L was distilled from Wan2.1-14B into a two-step causal model following the Causal Forcing++ recipe, and Apple released the full chain of training checkpoints: the causal teacher, the consistency-distilled model, the DMD-distilled model, and the final generator with its pixel decoder and KV refinement network. FastFrames-S keeps its two-step frame-wise Causal Forcing++ generator frozen and trains the networks around it, building on the Wan2.1-T2V-1.3B base. Releasing intermediate checkpoints matters for practitioners, because it lets a team start from an earlier stage instead of repeating the whole distillation.
The release also includes an optimized version of LingBot 1.3B. The model card says it runs 16x faster than the official model and supports fluid camera-conditioned control, which is the clearest concrete demo of what interactive latency buys: you move a camera and the generated world keeps up.
Output Specs, Setup and License
Generation output is fixed at 832 x 480 resolution, 81 frames at 16 fps. The setup requirements from the model card are narrow and worth knowing before you plan a test: Linux x86-64 with an NVIDIA GPU and a CUDA 12.8 compatible driver. Installation is a single command from the repository root:
uv sync
After downloading the checkpoints through the checkpoint guide, text-to-video generation runs with one command. Swapping the model flag selects the large variant:
uv run python scripts/generate.py --model ff-s --checkpoint-dir checkpoints \
--prompt "A corgi surfing a wave at sunset" --seed 0 --output outputs/corgi.mp4
The latency numbers are reproducible with the bundled benchmark, which is the right way to test the claim on your own hardware:
uv run python evaluation/latency/benchmark.py ff-s
uv run python evaluation/latency/benchmark.py ff-l --gpus 8
Licensing is split. FastFrames code is under the Apple Sample Code License, the model cards list the weights under Apple AMLR terms, and the LingBot-World v2 code and weights follow CC BY-NC-SA 4.0, which blocks commercial use of that part. Teams evaluating this for a product should separate the FastFrames models from the LingBot demo path before assuming the whole release is commercially usable.
Limits and Unverified Claims
Every number in this release is the authors' own measurement. No independent reproduction has been confirmed yet, and the quality claim rests on the paper's own VBench evaluation, which the model card points readers to reproduce. Three limits are explicit in the sources. First, all latency figures are B200 figures: no latency has been published for consumer GPUs or Apple Silicon, which is an unusual gap for an Apple release and the first thing most readers will want measured. Second, 480p is the current published ceiling, and behavior at higher resolutions is unconfirmed. Third, interactive latency at 36 ms per frame still depends on the full serving stack around the model, and the paper's applications are camera-conditioned and physics-conditioned generation, not a shipped product.
None of that cancels the profiling result, which is the durable part of this release. Even if the exact frame times shift on other hardware, the finding that decoding and cache updates can exceed half of end-to-end latency is a measurement any video team can repeat on its own pipeline this week, with code and checkpoints already public.
FAQ
What is Apple FastFrames?
FastFrames is a family of autoregressive two-step video diffusion models released by Apple on Hugging Face. FastFrames-S has 1.3B parameters and FastFrames-L has 14B, and both generate 480p video at interactive latency.
Why is FastFrames fast?
The authors profiled state-of-the-art pipelines and found that VAE decoding and KV cache updates account for over half of end-to-end latency. FastFrames replaces them with a lightweight pixel decoder and a KV refinement network, which cost 0.9 ms and 2.2 ms per frame in the published FastFrames-S breakdown.
What hardware do the latency numbers need?
FastFrames-S runs at 36 ms per frame on a single NVIDIA B200 GPU. FastFrames-L runs at 45 ms per frame on an 8x B200 node. The authors have not published latency for consumer GPUs or Apple Silicon.
How do you run FastFrames locally?
The model card requires Linux x86-64 with an NVIDIA GPU and a CUDA 12.8 compatible driver. You install with uv sync, download the checkpoints from the checkpoint guide, and generate with scripts/generate.py using the ff-s or ff-l model flag.