Power Planning for NVIDIA GPU Racks: H100 to GB200

Why GPU Rack Power Planning Is a Different Animal

Traditional enterprise data-center design rarely asked a facilities team to provision more than 10–15 kW per rack. NVIDIA's current GPU platforms have shattered that baseline. An eight-GPU DGX H100 node draws approximately 10.2 kW — meaning a single fully populated rack of H100 nodes can already push well past what conventional raised-floor infrastructure was designed to support. Move to the GB200 NVL72 platform and you are looking at roughly 120 kW from a single rack. For data-center builders and cabling contractors, this is not an incremental upgrade problem; it is a ground-up infrastructure rethinking problem.

This article walks through the power, cooling, and structured cabling decisions that must be made before a single GPU is racked — covering the H100 and H200 generation through the Blackwell B200 and GB200 NVL72.

Understanding the GPU Platforms and Their Power Profiles

Before you can size electrical and mechanical infrastructure, you need to understand what is actually being installed. NVIDIA's current GPU lineup breaks into two broad generations with meaningfully different facility requirements:

• H100 (Hopper, SXM5): 700 W TDP per GPU, 80 GB HBM3. In an eight-GPU DGX H100 configuration, total node draw is approximately 10.2 kW. Air cooling remains viable at this density, though rear-door heat exchangers are strongly recommended once rack densities climb.
• H200: Shares the same SXM5 socket and 700 W TDP as the H100, but steps up to 141 GB HBM3e memory. From a power-planning standpoint, H200 deployments can largely reuse H100 electrical infrastructure — the GPU socket power envelope does not change.
• B100/B200 (Blackwell): TDP rises to the 700–1,000 W range per GPU with up to 192 GB HBM3e and NVLink 5.0 delivering 1.8 TB/s interconnect bandwidth. The higher thermal envelope means air cooling is effectively off the table for dense Blackwell deployments.
• GB200 NVL72: This is the step-change that redefines the design brief. Seventy-two Blackwell GPUs combined with 36 Grace CPUs in a single liquid-cooled rack draws approximately 120 kW. This is not a typo. At this density, every infrastructure layer — power, cooling, cabling, and physical structure — must be purpose-engineered.

Power Distribution: Getting Electricity to the Rack

At H100 densities, many facilities can adapt existing 208 V or 480 V 3-phase circuits with appropriate PDUs. At GB200 densities, 480 V 3-phase distribution is the practical minimum, and overhead busway systems become strongly preferable over traditional raised-floor power distribution. Raised-floor plenum approaches that worked for 5–10 kW racks create cable management and airflow nightmares when you are running liquid cooling hoses alongside high-current whips in the same constrained pathway.

Overhead busway systems — where tap boxes drop power vertically to the rack — keep the floor clear for liquid cooling supply and return manifolds, preserve serviceability, and allow circuit reconfigurations without major civil work. For GB200 deployments specifically, work with your electrical engineer early to confirm that the facility's transformer capacity, switchgear ratings, and upstream utility service can actually absorb what a multi-rack AI cluster demands. It is not unusual for a modest cluster of GB200 racks to require a dedicated utility transformer.

UPS topology matters too. N+1 redundancy at minimum, with careful attention to runtime calculations — GPU training jobs do not checkpoint gracefully on a 10-millisecond power event. Discuss extended runtime modules or flywheel-based ride-through systems with your client when the workload is continuous training rather than inference.

Cooling: Air Is Not Enough Above a Threshold

The inflection point is clear in the hardware specs. H100 and H200 deployments can operate with precision air cooling supported by rear-door heat exchangers. Once you cross into Blackwell territory — and certainly at GB200 NVL72 densities — direct-to-chip liquid cooling is required, not optional. NVIDIA's own design mandates liquid cooling for the GB200 NVL72 rack.

For contractors, this creates a significant coordination requirement. Liquid cooling infrastructure — supply and return manifolds, leak detection systems, quick-disconnect fittings at the rack, and facility chilled-water or cooling distribution unit (CDU) loops — must coexist physically with structured cabling pathways. High-count fiber trunk bundles and liquid cooling hoses have very different bend-radius and weight requirements, and running them in the same overhead ladder rack without deliberate separation planning is a recipe for installation problems and future maintenance headaches.

Structured Cabling: High-Density Fiber Is the Norm

GPU clusters at any density generate enormous cabling volumes. Within a DGX or HGX node, NVLink cabling is handled via proprietary baseboard harnesses — contractors do not touch those. What contractors do build is the inter-node fabric: InfiniBand (ConnectX-7 at 400 Gb/s NDR with Quantum-2 switches) or Ethernet via NVIDIA Spectrum-X (400 GbE with RoCEv2, adaptive routing, and congestion control).

At 400G per port and above, the transceiver and cable selection rules are straightforward in principle but demanding in practice:

• DAC (Direct Attach Copper): Appropriate for connections of 3 meters or less. Cost-effective and low-latency, but adds significant weight and stiffness in high-port-count deployments.
• AOC (Active Optical Cable): The workhorse for 3–30 meter runs within a row or between adjacent rows. QSFP-DD and OSFP form factors dominate at 400G; QSFP-DD800 is emerging for 800G links.
• Structured fiber: For longer runs back to a top-of-row or end-of-row aggregation switch, pre-terminated multimode or single-mode fiber trunks based on 144-fiber MPO/MTP connectors are standard. Installations must comply with ANSI/TIA-568.3-D for optical fiber cabling and follow the bend-radius and connector-loss requirements documented there.

Copper horizontal cabling, where it appears (out-of-band management, BMC/IPMI, KVM), falls under ANSI/TIA-568.2-D. All infrastructure — fiber, copper, and pathway — in the data center should be planned and documented in alignment with ANSI/TIA-942 for data center infrastructure and administered per ANSI/TIA-606-C for labeling and records. In a GPU cluster with thousands of ports, labeling discipline is not a luxury; an unlabeled 400G port is a multi-hour troubleshooting event waiting to happen.

Fat-Tree and Rail-Optimized Topologies

AI clusters commonly deploy fat-tree or rail-optimized switch topologies. Both approaches require careful upfront cable planning because the inter-switch cabling volume is high and the physical routing between spine and leaf layers must be accounted for in pathway fill calculations. Plan overhead cable trays with realistic fill margins — 40% fill maximum is a sound rule of practice — and document the routing of every trunk before installation begins.

Coordination Is the Real Deliverable

The technical specifications for H100 through GB200 infrastructure are demanding, but the harder challenge is coordination. Electrical, mechanical, and low-voltage trades must work from a unified model. The liquid cooling contractor's manifold routing affects the cabling contractor's tray layout. The power distribution busway position affects how overhead fiber trunks are routed. The UPS and generator sizing affects the electrical engineer's panel schedule, which in turn affects circuit labeling under TIA-606-C.

For data-center builders deploying NVIDIA GPU infrastructure at any density above H100 single-node, the recommendation is the same: bring the structured cabling, power, and cooling teams to the table before the first conduit stub is set. The cost of rework at 120 kW per rack is not measured in parts — it is measured in project weeks.

Todd Taskerud, RCDD, is a structured cabling and data-center infrastructure specialist at Heather Technologies.