PyTorch 2.15 speeds Mac MoE decoding up to 6.5×

PyTorch 2.15 brings large speedups and new capabilities to Apple silicon Macs. The release is from the PyTorch team at Hugging Face, which works on the MPS backend to make the Mac a first-class place to run and train PyTorch models. At the start of 2026, running even small models on MPS often meant working around unsupported operations, performance cliffs and unexpected memory growth. The team runs real models from the Hugging Face Hub across language, image and video, finds where MPS falls short, and fixes it upstream in PyTorch, increasingly by replacing MPSGraph with dedicated Metal kernels.

The headline result is for mixture-of-experts (MoE) language models. Mixture-of-experts models decode 5–6.5× faster than in 2.14. On an M3 Ultra with torch.compile, Qwen3.6-35B-A3B goes from 9.4 to 61.3 tokens/s, and LFM2.5-8B-A1B goes from 23.6 to 123.6 tokens/s. Dense models benefit too, from faster linear layers and an 8× faster softmax for sampling.

Faster language-model decoding

Every LLM has a prefill phase and a decode phase. During decode, each new token passes through the model as a single row, so large matrix multiplications become matrix-vector products. PyTorch 2.15 speeds up both kinds of layers that dominate this step: dense linear layers and mixture-of-experts layers.

PyTorch 2.14 had GEMV kernels for torch.mm, but F.linear still went through MPSGraph. In 2.15, F.linear uses them too, so the common [1, 1, hidden_size] decoding input runs on the dedicated kernels, with eligible bias additions fused in. Across 96 BF16 layer shapes from real models, this is a 1.23× geometric-mean speedup. The biggest wins land on small projections, such as the gates in MoE models, where the old path left most of the memory bandwidth unused. Large layers and LM heads were already close to bandwidth-bound, so they stay near 1×.

Mixture-of-experts layers send different numbers of tokens to each expert. Grouped matmul handles those uneven groups in a single operation, and 2.15 adds a native MPS implementation. In 2.14, grouped matmul copied the group offsets to the CPU and looped over the groups. The new Metal kernels read the offsets directly on the GPU, using Metal Performance Primitives where available and a separate Metal path elsewhere. Small groups, the common case during decoding, get their own kernel as well. The article's example splits 64 tokens into groups of 16, 32 and 16 and calls F.grouped_mm with offs= set to the group end offsets.

FP8 comes to the Mac

PyTorch 2.15 adds torch.float8_e4m3fn tensors on MPS. Each value takes one byte, half the storage of FP16 or BF16. You can multiply these tensors with F.scaled_mm, which supports both tensorwise and rowwise scaling. The article's example quantizes FP32 inputs to FP8 with one scale per tensor and multiplies them with F.scaled_mm, producing BF16 output.

Linear algebra gets a boost

Complex tensors are easier to use. complex64 LU factorization, LU solve and triangular solve now run on MPS. This also unlocks operations and gradients built on them, such as complex determinants.

Small matrices get fast kernels too. For camera geometry and image transformations, batches of FP32 matrices up to 4 × 4 are inverted in a single Metal kernel that combines factorization and solving. For larger problems, QR uses blocked updates, and triangular solves turn much of their work into efficient matrix multiplication. Gated DeltaNet workloads get a dedicated fast path for batches of 16 × 16, 32 × 32 and 64 × 64 FP32 triangular systems with at least eight right-hand sides.

SVD is more robust at numerical extremes. It now handles very small singular values, very large or very small input magnitudes, and basis completion with full_matrices=True. This makes pseudoinverse and least-squares results more reliable.

Convolutions, padding and training

Audio models, video decoders and training loops get new Metal paths for several operations: conv1d forward, constant padding, reflection and replication padding, upsampling backward, and channels-last conv2d backward. Out-of-place dropout is also covered: nn.Dropout and F.dropout now use the existing fused kernel.

Softmax and reductions

Token sampling gets faster. Softmax and log_softmax now run on dedicated Metal kernels, and large vocabularies benefit most. An FP32 softmax over a [1, 128256] row of logits drops from 57 µs to 7 µs, about 8× faster than the MPSGraph version. The article says the speedup holds across other common vocabulary sizes.

A much smaller memory footprint for video decoding

The constant-padding change has one of the most striking results. The VAE decode in Wan-AI/Wan2.2-TI2V-5B-Diffusers makes 8,764 padding calls over a full 28-tile decode. With padding on Metal, peak driver memory drops from about 270 GiB to 8 GiB. The tensor-allocation peak is 4.422 GiB with MPSGraph padding and 4.095 GiB with Metal padding. The allocator-reserved peak is 8.070 GiB in both cases. The total driver-memory peak is 270.135 GiB with MPSGraph padding and 8.108 GiB with Metal padding.

The same padding change also fixes a correctness bug. On 2.14, the Qwen-Image-2.1 VAE encoder silently corrupted images on MPS, breaking image-edit workflows reported in ComfyUI. An encode-decode round trip now matches CPU at 29.95 dB PSNR, up from 10.06 dB.

More control with MPS streams

You can now choose which MPS stream runs your work from Python. torch.mps.Stream and the torch.mps.stream context manager provide the interface, and Tensor.record_stream lets the allocator track a tensor's use on another stream. The article's example sends work to a separate stream, calls record_stream, and synchronizes before reading the result.

Several more operations now run on MPS, including torch.frexp and torch.logspace.

Requirements and getting started

PyTorch 2.15 requires macOS 15 or later and Python 3.11 or later, and MPS now requires macOS 15 or later. You need an Apple silicon Mac, and the 2.15 compatibility matrix lists the supported combinations. If your OS is too old or no Metal GPU is available, the MPS availability check now reports the reason.

A sneak peek at PyTorch 2.16

The team plans attention fully on Metal, more MPSGraph-to-Metal migrations, and faster LLM decoding in PyTorch 2.16.

Key Features at a Glance

• Mixture-of-experts decoding: 5–6.5× faster than 2.14 – Qwen3.6-35B-A3B rises from 9.4 to 61.3 tokens/s on an M3 Ultra, so local chat with large MoE models becomes much more responsive.
• FP8 tensors: one byte per value, half the storage of FP16 or BF16 – lets you hold large weights in less memory and multiply them with F.scaled_mm.
• Dense linear layers: 1.23× geometric-mean speedup across 96 BF16 layer shapes – helps the smaller projections that dominate token generation.
• Softmax for sampling: about 8× faster, 57 µs to 7 µs on a [1, 128256] FP32 row – speeds up picking the next token over large vocabularies.
• Video VAE decode: peak driver memory from about 270 GiB to 8 GiB – makes Wan2.2 tiled decoding practical on a Mac.
• Qwen-Image-2.1 encoder fix: 10.06 dB to 29.95 dB PSNR against CPU – restores correct output for image-edit workflows.
• Complex linear algebra: LU factorization, LU solve and triangular solve for complex64 – enables operations such as complex determinants on the GPU.
• MPS streams: torch.mps.Stream and torch.mps.stream in Python – lets you direct work to a chosen stream.

Availability

The release is a release candidate. To try it in a separate environment, the article gives this command:

uv pip install torch==2.15.0 --index-url https://download.pytorch.org/whl/test/cpu

PyTorch 2.15 requires macOS 15 or later and Python 3.11 or later. The article does not give a final release date.