Cluster Layer

Racks joined into one computer · Scale-up, scale-out, scale-across · Flagship: AWS Project Rainier

A cluster joins thousands of racks into one computer at three scales: inside the rack, across the building, and between sites. The largest now hold about half a million accelerators, from SpaceX's Colossus 2 to AWS's Trainium-based Project Rainier, and Microsoft and Google train across data centers hundreds of miles apart.

Three scales of an AI cluster: scale-up inside a 72-GPU NVLink rack, scale-out across a building on 800 Gbps Ethernet or InfiniBand, and scale-across between sites such as Microsoft's Fairwater campuses 700 miles apart; and the largest clusters by accelerator count, from Meta's 16,384-GPU Llama 3 run to about 550,000 GPUs at SpaceX Colossus 2 THREE SCALES OF ONE CLUSTER SCALE-UPOne rack, 72 GPUsNVLink, 130-260 TB/s SCALE-OUTOne building, 100,000sEthernet or InfiniBand SCALE-ACROSSMany sitesAI WAN, 700 miles LARGEST CLUSTERS, ACCELERATORS (LOG SCALE) 1K 10K 100K 1M SpaceX Colossus 2GB200 and GB300, 2026550,000 AWS Project RainierTrainium2, Indiana500,000 SpaceX Colossus 1Hopper, 2024100,000 Google Ironwood podOne TPU pod9,216 Meta Llama 3 runH100, 202416,384 A failed NVLink part takes down its rack;scale-out networks route around failures.
NVL72 domain
72 GPUs
Largest clusters
~500,000+
Fairwater sites apart
~700 miles
Multi-site goodput
93% (Google)
Hub
Stack

What it is

Thousands of Racks Acting as One Computer

A cluster is a set of racks joined by networks fast enough to train one model together. It works at three scales.

  • Scale-up: inside a rack, NVLink joins 72 GPUs so they share memory; NVLink 5 moves 130 TB/s across a GB300 NVL72 and NVLink 6 doubles that in Vera Rubin.
  • Scale-out: across a building, a back-end network of 800 Gbps Ethernet or InfiniBand joins hundreds of thousands of GPUs. A separate front-end network handles storage and users.
  • Scale-across: between buildings or sites. Google trained Gemini 2.5 across several data centers at 93% goodput; Microsoft links its Fairwater sites 700 miles apart.
  • Failures: a failed NVLink part takes its whole rack out of the job; scale-out networks route around failed links. Meta's 16,384-GPU Llama 3 run had 419 unexpected interruptions in 54 days. See AI Training.

Flagship

AWS Project Rainier: A Cluster Spread Across Buildings

  • Size: nearly 500,000 Trainium2 chips for Anthropic in New Carlisle, Indiana, the largest documented non-Nvidia cluster.
  • Scale-up: 64-chip UltraServers spanning two racks, joined by NeuronLink-v2 over copper.
  • Scale-out: AWS's Elastic Fabric Adapter (EFA) Gen 3 in a design AWS calls 10P10U: tens of petabytes per second across the cluster and about 10 microseconds between any two chips, with multipath routing around failed links and switches.
  • Across buildings: one cluster spans multiple buildings on the campus. See AWS Project Rainier.

Largest

The Largest Clusters in 2026

  • SpaceX Colossus 2: about 550,000 GB200 and GB300 GPUs in Memphis by Musk's count; its network has not been published. Colossus 1 ran 100,000 Hopper GPUs on Nvidia Spectrum-X Ethernet.
  • Microsoft Fairwater: Satya Nadella describes hundreds of thousands of GB200s in one Wisconsin cluster, on two-tier 800G Ethernet.
  • China: Huawei's CloudMatrix 384 joins 384 Ascend chips with an all-optical mesh, about 300 petaflops at about 559 kW, about 2.3 times less efficient per operation than NVL72. Tencent's Xingmai 2.0 network supports more than 100,000 GPUs.

Clusters

AI Clusters

AI clusters and their fabrics
ClusterOperatorAcceleratorsScale-upScale-outStatus
SpaceX Colossus 2SpaceXSI~550,000 GB200 and GB300NVL72Not disclosed (Colossus 1: Spectrum-X Ethernet)Operating
Project RainierAWS for Anthropic~500,000 Trainium264-chip UltraServer, NeuronLink-v2EFA Gen 3, ~10 microseconds between any two chipsOperating
Fairwater Wisconsin and AtlantaMicrosoftHundreds of thousands of GB200 and GB300NVL72Two-tier 800G Ethernet on SONiC; AI WAN between sitesOperating
Ironwood podsGoogle9,216 TPUs per pod3D torus with optical circuit switchesJupiter network; multi-pod jobsOperating
Huawei CloudMatrix 384Huawei384 Ascend 910C in 16 racksAll-optical mesh, 6,912 transceiversOperating
Huawei Atlas 950 SuperPoDHuawei1,024 Ascend 950UnifiedBus 2.0Shown July 2026
Tencent Xingmai 2.0Tencent100,000+ GPUsTencent's own networkOperating

Record

Dataset Record

Dataset
AI clusters: scale-up, scale-out and scale-across designs
Sources
Nvidia, AWS, Microsoft, Google (IEEE Micro 2026), Meta, Huawei, Tencent, Epoch AI, SDxCentral, Data Center Frontier, DatacenterDynamics, Tom's Hardware
Last verified
October 2026