Amazon launches TorchNeuron, a native PyTorch backend that runs existing PyTorch code on AWS Trainium2/3 with a one-line device change; open-source (Apache 2.0) public beta lands Oct 20 at PyTorch Conference

Why it matters

【Key points】

Amazon announced TorchNeuron on Oct 19, first on the PyTorch blog (20:45 CST) and then on aboutamazon.com (21:00 CST). It is a native PyTorch backend that makes AWS Trainium a standard PyTorch device: existing code runs with .to('neuron') and no rewrite. It will be open-sourced under Apache 2.0 (github.com/aws-neuron/torch-neuronx) as a public beta at PyTorch Conference North America on Oct 20.

According to Amazon, it plugs in through PyTorch's PrivateUse1 extension point. It supports eager mode (plus an experimental adaptive eager mode), torch.compile (lowered via Torch-MLIR to StableHLO, then the Neuron compiler), FlexAttention and NKI custom kernels. Distributed support covers DDP, FSDP2, tensor and pipeline parallelism, and DTensor, with collectives on Trainium's collective cores and EFA. BF16/FP32/FP16 work; FP8 and TorchAO are private beta. It covers Trainium2 and Trainium3; vLLM Neuron plugins, Hugging Face tools and TorchTitan run on it.

Early users include Cartesia and the Lieber Institute. Amazon says each build is tested on hundreds of thousands of models and it is upstreaming fixes that remove chip-specific code from PyTorch.

【Verifiable】

Amazon: https://www.aboutamazon.com/news/innovation-at-amazon/torchneuron-pytorch-aws-trainium-chips-ai

PyTorch blog: https://pytorch.org/blog/introducing-torchneuron-native-pytorch-backend-for-aws-trainium/

【Our read】

1. Nvidia's moat is CUDA as much as silicon. Making Trainium a native PyTorch device, instead of requiring Neuron-specific code, cuts the cost of switching away from GPUs.

2. "No code change" is a vendor claim. Performance, op coverage and compile stability need community testing after the Oct 20 release, and FP8 isn't GA yet.

3. Google's TPUs and AMD's ROCm are racing for the same thing: native PyTorch backend status. A more hardware-agnostic PyTorch makes in-house cloud chips easier to sell.