AWS and Hugging Face have detailed a technical breakdown of cloud architecture for foundation models, featuring P6 instances equipped with NVIDIA Blackwell GPUs capable of reaching up to 13.5 PFLOPS of dense FP4 compute per GPU. The analysis spans multi-node compute, petabit-scale Elastic Fabric Adapter (EFA) networking, and tiered storage to meet convergent pre-training, post-training, and test-time compute requirements. Machine learning engineers can leverage these concrete specifications to diagnose distributed system bottlenecks across large-scale model deployments.
Next-Gen Blackwell Compute Specs
AWS outlined hardware specifications across several generations of accelerated instances, highlighting a transition from NVIDIA H100 and H200 accelerators to Blackwell-based architectures:
- p5.48xlarge: 8x H100 GPUs, 640 GB HBM3, 0.9895 dense BF16 PFLOPS per GPU, 7.2 TB/s NVLink, and 400 GB/s EFAv2 networking.
- p5en.48xlarge: 8x H200 GPUs with 1,128 GB HBM3e and EFAv3, reducing packet latency by approximately 35% compared to EFAv2.
- p6-b200.48xlarge: 8x B200 GPUs delivering 2.25 dense BF16 PFLOPS, 4.5 FP8 PFLOPS, and 9 FP4 PFLOPS per GPU, backed by 1,440 GB HBM3e and 14.4 TB/s aggregate NVLink bandwidth.
- p6-b300.48xlarge: 8x B300 GPUs providing up to 13.5 PFLOPS of dense FP4 throughput per GPU, 2,100 GB total HBM3e memory, and 800 GB/s aggregate EFAv4 networking.
Each iteration addresses the expanding memory and interconnect demands of foundation model workloads, preventing training steps from stalling during intensive operations.
Scale-Out Interconnects and EFA Architecture
To manage collective communications like all-reduce and all-gather across instances, AWS pairs internal NVLink fabrics with external EFA networking. EFA relies on the Scalable Reliable Datagram (SRD) protocol via the Libfabric API, bypassing the host operating system kernel to minimize latency across multi-node jobs.
EFAv4, deployed on P6 instances, delivers an 18% collective communication performance improvement over EFAv3. When deployed across thousands of nodes within Amazon EC2 UltraClusters, this nonblocking networking layer ensures high-bandwidth communication across distributed clusters.
Tiered Storage and Extended UltraServers
For workloads requiring expanded NVLink domains, AWS detailed P6e-GB200 UltraServers built on the NVIDIA GB200 NVL72 platform. Component instances like the p6e-gb200.36xlarge offer four GPUs with 740 GB HBM3e and 200 GB/s EFAv4 bandwidth. When combined into an UltraServer configuration, up to 72 Blackwell GPUs and 13.4 TB of aggregate HBM3e are linked within a single NVLink domain using Grace-Blackwell superchips.