TorchTPU: Native PyTorch Backend for TPUs, 52% MFU Claimed
At PyTorch Conference 2026 on October 20, TorchTPU shipped as open source. It is a native PyTorch backend for TPUs, and the core claim is that existing PyTorch model and training code keeps running as-is. Google's vice presidents Mark Lohmeyer and Bill Jia introduced it on the official Google Cloud blog (October 20). A PyTorch-paradigm introduction ran on the pytorch.org blog the same day.
One-line summary for developers: Google and Meta's PyTorch teams co-developed TorchTPU over the project Google started in mid-2025. Marktechpost confirms the license is Apache 2.0 (October 20). The design explains itself as a bill for the four problems with PyTorch/XLA from 2020. The second news is that Meta uses it in production: its stated position is that it "cannot be locked into a single hardware architecture."
The Design Concept: Eager First, Plus a Compile Path
- Eager execution comes first: no compilation at first execution, so code runs instantly. TorchTPU is an unofficial out-of-tree backend built on PyTorch PrivateUse1.
- A compile path is provided alongside: TorchDynamo lowers to StableHLO, which then goes through XLA.
- Detection is split: debug with eager mode, ship production with
torch.compile. The PyTorch blog does not describe eager and compile as a binary choice but always as a pair. - The migration story stays minimal: move the device from
.to("cuda")to.to("tpu")and leave the rest essentially untouched:
device = "tpu" # was "cuda"
model = build_model().to(device)
for batch in loader:
out = model(batch.to(device))
loss = criterion(out)
loss.backward()
The Numbers: Qwen 3 8B at 52% MFU via TorchTitan
- Up to 52% MFU: on a Qwen 3 8B training run via TorchTitan.
- Near-linear scaling to 8,192 Ironwood TPUs: the figure behind the "at NVIDIA scale" framing in the official announcement.
52% MFU is Google's announcement figure, not independent verification. Keywords to watch afterward are MFU on longer sequences and the variance of scaling efficiency by model size.
Four Problems With the Previous Path: PyTorch/XLA (2020)
The official announcement lists the existing path's issues, and they double as the design rationale for the new backend:
- Invasive source edits: explicit compilation regions had to be defined.
- Severe compile latency: heavy compile lag at first execution.
- No eager execution support: debugging or exploratory iteration followed the compile-first path directly.
- Distributed primitives and features lagged behind: keeping up with the PyTorch API's evolution was hard.
TorchTPU claims to solve all four: "eager first, compile as a production switch" (blog highlights, October 20).
The Ecosystem Story: vLLM, SGLang, LiteRT, One-Line Hugging Face Deployment
What reads bigger than the performance number is the ecosystem story. On the serving side sit vLLM-TorchTPU and SGLang-TorchTPU. Integration with llm-d, TorchTitan, Ray, TorchRec, VeRL, and Diffusers was announced alongside. llm-d 1.0 (#3369) shipped at the same conference.
- Hugging Face Transformers v5 deploys to TPUs in one line: with
device_map="tpu", HF models run across platforms without touching the backend (webpronews, October 20). - LiteRT (formerly TFLite) joins the PyTorch ecosystem.
- Meta is already a production user. Jana van Greunen, Meta's PyTorch engineering director, stated they "cannot be locked into a single hardware architecture". LLM, advertising, and recommendation workloads have reportedly run on the new backend (unite.ai, October 20).
- The TPU agent suite (an inference framework on NativeCUDA APIs) appears in public preview on GitHub.
# Hugging Face — deploy to a TPU in one line
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained(name, device_map="tpu")
Extent of Meta's use is undisclosed: Meta production use is likely limited to a subset of workloads rather than the whole stack. Public ranking for the Ironwood TPU is not confirmed. TorchTPU is real open source (Apache 2.0, with review by community contributions). As an out-of-tree backend, its long-term maintenance participatory model is an open question.
Practical Takeaways: Selection Criteria for Tool Builders
- "Eager first, compile optional": a backend template for Axion custom silicon: hit debugging speed and production performance at once without force-feeding compile to modelers.
- PrivateUse1 is a handoff trunk. Tools that must run across backends (vLLM, llm-d, Diffusers) argue explicitly for opaque unified APIs rather than claiming PyTorch native support immediately.
- One-line Hugging Face deployment: as a listing distribution template: for a new accelerator, get arbitrary teams calling
device_map="X"within 48 hours of release.
Sources
- Original issue #3377
- Google Cloud Blog: TorchTPU: Running PyTorch natively on TPUs (Mark Lohmeyer, Bill Jia · October 20)
- PyTorch Blog: TorchTPU: Extending PyTorch to TPUs
- Related issue #3369: llm-d 1.0 announced at the same conference
FAQ
What is TorchTPU?
TorchTPU is a native PyTorch backend for TPUs that shipped as open source at PyTorch Conference 2026 on October 20. Google and Meta's PyTorch teams co-developed it over the project Google started in mid-2025, and the license is Apache 2.0. Its core claim is that existing PyTorch model and training code keeps running as-is.
What code changes are needed to run a model on a TPU?
The migration story stays minimal: move the device from .to("cuda") to .to("tpu") and leave the rest essentially untouched. Eager execution comes first, with a compile path provided alongside for production through torch.compile. Hugging Face Transformers v5 also deploys models to TPUs with device_map="tpu" in one line.
What performance does TorchTPU claim?
The announcement claims up to 52% MFU on a Qwen 3 8B training run via TorchTitan, and near-linear scaling to 8,192 Ironwood TPUs. The 52% MFU figure is Google's announcement figure, not independent verification.
Who uses TorchTPU in production?
Meta is already a production user, and its LLM, advertising, and recommendation workloads have reportedly run on the new backend. The extent of Meta's use is undisclosed. Production use is likely limited to a subset of workloads rather than the whole stack.