NVIDIA announced official support for eight-node DGX Spark clusters at GTC Berlin, scaling unified memory to 1 TB. Each DGX Spark provides 128 GB of unified memory, so four nodes reach 512 GB and eight nodes reach 1 TB. That capacity opens the door to models that previously wouldn't start on a four-node system, including the full GLM 5.3 (753B) model from Z.ai. The benchmarks show that eight nodes deliver the memory headroom and near-linear training scaling, while the best network topology depends on the workload. Support for eight-node clusters in NVIDIA Sync is coming by the end of the year, and NVIDIA Sync's Cluster Assistant handles setup and device management.

Key Results at a Glance

• Memory: 1 TB of unified memory across eight nodes, up from 512 GB at four nodes and 128 GB on a single Spark.
• Full GLM 5.3 (753B) decode: 31 TPS for one user, 51 TPS total at concurrency four, and 67 TPS total at concurrency eight.
• GLM 5.3 Flash generation: roughly 80 TPS on an eight-node cluster, suitable for interactive local development.
• ROCEnante decode scaling: 4.43x at eight Spark nodes versus 3.05x without it, a 45% improvement in scaling efficiency on a switch topology.
• Request routing: a routed agent scored 91/100 on the 92-case tool-use benchmark, matching the Qwen-only configuration, with about 1.5x to 3.0x aggregate throughput.
• Nanochat pre-training: 7.79x speedup at 97.4% scaling efficiency on eight nodes.

Memory for Frontier-Class Models

AI developers increasingly run several models in parallel, chain inference calls, and experiment with frontier-class architectures that need hundreds of gigabytes of memory. The biggest benefit of an eight-node cluster is memory. The cluster is also designed to serve these models interactively, not just load them.

The full GLM 5.3 (753B) checkpoint runs across the 8x 128 GB cluster, making it one of the largest open models you can serve locally. The model is nvidia/GLM-5.3-NVFP4, about 470 GB in NVFP4. As an early example, NVIDIA ran GLM 5.3 Flash across an eight-node cluster and measured roughly 80 tokens per second (TPS) in generation, which NVIDIA describes as suitable for interactive local development against a frontier-class mixture-of-experts model.

Serving the full model at concurrency one, four, and eight produces the following results, measured with MTP3 and 8K input sequence length (ISL) per request for prefill:

• Prefill: 1,600 TPS total at every concurrency level, which works out to 1,600 TPS per user at concurrency one, 750 at four, and 473 at eight.
• Decode: 31 TPS total and per user at concurrency one; 51 TPS total and 21 per user at four; and 67 TPS total and 18 per user at eight.

Faster Collectives with ROCEnante

Tensor-parallel inference spends a meaningful share of its time in collective operations such as all-reduce. On switch-connected clusters, NVIDIA is evaluating a Spark-optimized collective backend called ROCEnante. It targets the small-message, low-concurrency, modest-rank-count pattern typical of tensor-parallel decode on local AI hardware.

The backend implements an all-to-all exchange followed by local reduction. Each rank shares its input with every other rank, then adds the inputs locally, and a single GPU kernel coordinates the exchange and reduction. The main benefit is fewer communication steps. Ring all-reduce passes data between neighboring ranks over multiple rounds, while this approach exchanges data directly with all peers, avoiding that chain of forwarding. For small messages and a modest number of ranks, this can reduce latency.

The tradeoff is more data movement. Each rank sends its full input to every peer and reduces all the inputs itself. As message sizes or the number of ranks grow, that extra traffic becomes more expensive, and ring all-reduce's bandwidth efficiency becomes more attractive. For high concurrency, larger clusters, or ring topologies, NVIDIA Collective Communications Library (NCCL) performs better.

In Figure 1, which uses Qwen3.8-27B-NVFP4 on a switch topology with speculative decoding disabled, decode speedup relative to one Spark starts at 1.00x for both configurations. At four Spark nodes, ROCEnante reaches 3.06x versus 2.56x without it. At eight nodes, it reaches 4.43x versus 3.05x. In practice, NVIDIA sees roughly 5% higher prefill and 21% higher decode throughput at four nodes with NVFP4, and about 18% higher decode throughput at eight nodes on the non-quantized model. The backend is merged into FlashInfer and falls back transparently to NCCL if its startup test fails. The original one-shot all-reduce approach was contributed by the community (Jason Cook/el8).



The backend applies only to switch-connected topologies, because it relies on direct point-to-point exchange between every node. It is not a replacement for NCCL, which remains the supported default and the better choice as cluster size and concurrency grow.

Ring or Switch: Choosing a Topology

Low-latency NVIDIA ConnectX-7 NICs and RDMA on Converged Ethernet (RoCE) let DGX Spark nodes exchange tensors efficiently, so you can scale a model across more nodes to fit it in memory and improve throughput. To isolate cluster scaling behavior, NVIDIA benchmarked a dense Qwen3.8-27B (NVFP4) model without speculative decoding or MTP across one to eight nodes. Measured with llama-benchy at 8192 tokens ISL, 256 tokens OSL, and concurrency one, the results are:

• One node: 2,635 TPS prefill and 11.86 TPS decode, the baseline.
• Two nodes (direct connect): 3,285 TPS prefill (1.25x) and 21.71 TPS decode (1.838x).
• Four nodes, physical ring: 4,446 TPS prefill (1.69x) and 32.82 TPS decode (2.77x).
• Four nodes, switch with ROCEnante: 4,556 TPS prefill (1.73x) and 36.34 TPS decode (3.06x).
• Eight nodes, physical ring: 4,962 TPS prefill (1.88x) and 40.11 TPS decode (3.38x).
• Eight nodes, switch with ROCEnante: 5,105 TPS prefill (1.94x) and 52.58 TPS decode (4.43x).

Two patterns stand out. First, decode throughput scales strongly with node count, from about 12 TPS on one node to 52 TPS on eight. Second, the best topology depends on the workload. A physical ring delivers excellent decode throughput for a single model spread across all nodes. A switch is needed once you want to split the cluster into independent groups, or run workloads that require direct any-to-any connectivity between nodes.

Not every cluster needs a switch. DGX Spark nodes can be wired directly together in a ring, which lowers cost and simplifies setup. Within a four-node deployment, NCCL defaults to ring-based algorithms for tensor-parallel communication, so a switchless ring aligns with traffic flows and offers strong throughput at lower expense. The picture changes as you scale or specialize:

• Eight nodes or split allocations: to split a cluster into, say, two groups of four, a ring cannot express that topology, so you need a switch.
• All-to-all and traditional HPC workloads: anything that depends on point-to-point patterns, such as MPI-based clustering, requires a switch, since a ring would pay a hop penalty.
• The optimized collective backend: ROCEnante requires direct point-to-point connectivity between every node, so it applies only to switched configurations.

In short, choose a switchless ring for the simplest, lowest-cost four-node inference box, and a switch when you need to scale to eight nodes, split the cluster, or run communication patterns a ring cannot serve. NVIDIA provides a networking guide that walks through cluster setup starting from a two-node configuration and scaling up.

Routing Requests with NeMo Switchyard

Not every request needs the largest model. On a shared eight-node cluster, routing easy requests to a small, fast model and reserving a larger model for hard ones can substantially increase aggregate throughput. NVIDIA NeMo Switchyard makes this practical: a lightweight classifier inspects each request or session and routes it to the appropriate model pool, with a load balancer distributing traffic across replicas. Because this layout splits the eight nodes into independent model pools, Switchyard runs on a switch-connected cluster, which provides the any-to-any connectivity needed to route requests across replicas.

In NVIDIA's prototype, OpenBMB's MiniCPM5 2B served as the classifier and the execution model for simple tasks, and Qwen3.8 Flash Next handled complex ones. On the 92-case tool-use benchmark, including hard cases, the routed agent scored 91/100, matching the Qwen-only configuration while freeing cluster capacity. MiniCPM serves roughly an order of magnitude more requests per second than Qwen per replica, so shifting part of the traffic to the smaller model raises the throughput ceiling of the whole cluster. With three Qwen replicas (two Spark nodes each) and two MiniCPM replicas (one Spark each), NVIDIA measured about 1.5x aggregate throughput over a Qwen-only baseline when half of requests route to Qwen, and about 3.0x when a quarter of requests route to Qwen and the rest go to MiniCPM. A routing policy is a simple YAML file, and NVIDIA provides a sample policy as a starting point that developers can tune for their own workload.

Training Scales Near-Linearly

Training behaves differently from inference. When each node holds a full copy of the model, the only cross-node communication is gradient synchronization once per step, so clustering scales close to linearly. NVIDIA pre-trained a nanochat model from scratch, about 896.5M parameters with 20 transformer layers, a hidden dimension of 1,280, and 2,048-token sequences, and measured throughput as nodes were added:

• 1 node: 8,741 TPS, 1.00x speedup, 100.0% efficiency.
• 2 nodes: 17,202 TPS, 1.97x speedup, 98.4% efficiency.
• 4 nodes: 33,640 TPS, 3.85x speedup, 96.2% efficiency.
• 8 nodes: 68