GPU Cost per kW_IT

Answer

The cost per kW_IT for current-generation AI accelerator hardware (GB200 NVL72) ranges from $25,000 to $40,000/kW_IT, with a central estimate of $32,500/kW_IT. This includes the full system cost (compute trays, NVLink switches, CDU, power distribution) but excludes data center shell, site infrastructure, and networking beyond the rack.

At hyperscaler volume pricing (~$3.1M per rack at 120 kW), the GPU hardware cost is ~$25,800/kW_IT. At all-in rack cost including networking and storage (~$3.9M), the figure rises to ~$32,500/kW_IT. For smaller buyers or fully burdened deployments including facility infrastructure, costs reach $40,000/kW_IT or higher.

Evidence

GB200 NVL72 rack economics

Source: semianalysis-gb200-tco. GB200 NVL72 rack ~$3.1M (hyperscaler price), ~$3.9M all-in. 120 kW/rack. TCO is ~1.6x higher than H100 per GPU. (SemiAnalysis, 2025)

Source: Introl, GB200 NVL72 deployment guide. "$3 million price tag" for the system. 120 kW continuous draw. Cooling capacity requirement of 2.4 MW (20x overprovisioning for cooling headroom). Four 30 kW power shelves, each requiring 480V three-phase input. 97% power conversion efficiency. (Introl blog)

Source: ChinaTalk, "How Much AI Does $1 Get You in China vs America?" Uses $2.6M per GB200 NVL72 rack and 145 kW per rack (including overhead) for modeling a 400 MW facility. At 90% IT allocation: 2,154 racks at $2.6M = $5.6B for hardware at a 400 MW facility. This yields ~$14,000/kW at total facility level, but this is the lower-bound hardware-only estimate. (ChinaTalk, Feb 2026)

Derived $/kW_IT calculations

Comparison with industry benchmarks

Source: McKinsey, "The cost of compute." Projects $5.2T total capex for 125 GW of incremental AI data center capacity (2025-2030). Of this, ~$3.5T is servers. This implies ~$28,000/kW_IT for servers alone ($3.5T / 125 GW) and ~$41,600/kW_IT all-in ($5.2T / 125 GW). (McKinsey, 2025)

Source: Thunder Said Energy, data center economics. Standard data centers cost ~$10,000/kW; AI-heavy computation reaches up to $40,000/kW, with "over half" being GPU hardware (~$20,000+/kW for servers). (Thunder Said Energy)

Source: Bernstein Research. Constructing a 1 GW AI facility costs ~$35B, implying $35,000/kW all-in including shell and site. NVIDIA leadership suggests $50-60B/GW for frontier AI facilities. (Bernstein, 2025)

SemiAnalysis TCO component breakdown

From SemiAnalysis InferenceX v2 analysis of compute tray content costs:

• H100 SXM compute tray: ~$170K
• B200 compute tray: ~$264K
• B300 compute tray: ~$344K
• MI300X compute tray: ~$138K
• MI355X compute tray: ~$197K

The trend shows GPU content cost increasing with each generation, driven by higher GPU pricing and NVIDIA's premium margins. (SemiAnalysis, Feb 2026)

Next-generation outlook: Vera Rubin

Source: SemiAnalysis, "Vera Rubin -- Extreme Co-Design." Vera Rubin NVL72 maintains the rack-scale Oberon architecture with 72 Rubin GPUs, 36 Vera CPUs, and 36 NVLink 6 Switch ASICs. Power envelope remains ~120-130 kW per rack. NVIDIA promises 5x inference performance and 10x lower cost per token vs Blackwell. This implies cost-per-FLOP drops dramatically, but cost-per-kW_IT may remain similar or increase slightly due to higher silicon content. (SemiAnalysis, Feb 2026)

Source: NVIDIA GTC 2026 analysis. Rubin NVL72 delivers 3.6 exaflops per rack (vs 1.4 for GB200 NVL72). Memory bandwidth increases 2.75x. HBM capacity increases 50% (192 GB to 288 GB per GPU). Production capacity limited to 200,000-300,000 units in 2026 due to TSMC packaging and HBM4 constraints. (tech-insider.org, Mar 2026)

Cooling costs increase modestly: $50K for GB300 NVL72, rising to $56K for VR NVL144. (Tom's Hardware / Morgan Stanley)

Analysis

Central estimate rationale

The central estimate of $32,500/kW_IT uses the all-in rack cost ($3.9M) divided by IT load (120 kW). This is appropriate because:

1. It includes compute hardware, NVLink switches, CDU, power distribution, and networking -- all components that must be launched for an orbital deployment comparison.
2. It excludes data center shell and site infrastructure, which would be replaced by spacecraft bus and solar arrays in an orbital scenario.
3. It aligns with McKinsey's server-only estimate of ~$28,000/kW_IT (the difference being McKinsey's figure is a fleet average including cheaper non-AI servers).

Optimistic and conservative bounds

• $25,000/kW_IT (optimistic): Hyperscaler volume pricing for the rack hardware itself, excluding networking and storage. Achievable by the largest buyers (Microsoft, Google, Meta) with long-term supply agreements and NVIDIA pricing concessions.
• $40,000/kW_IT (conservative): Fully burdened cost including rack infrastructure, facility-level power distribution, and cooling plant capital amortized per kW. Consistent with Thunder Said Energy's $40,000/kW estimate for AI-heavy facilities and Bernstein's $35,000/kW for a GW-scale facility.

Will Rubin change $/kW_IT?

Vera Rubin NVL72 delivers ~2.5x more compute per rack at a similar power envelope (~120-130 kW). However, the rack price is expected to increase roughly proportionally with compute content. SemiAnalysis data shows compute tray costs rising from ~$170K (H100) to ~$264K (B200) to ~$344K (B300), a trajectory that suggests VR NVL72 rack costs will be $4-5M+.

The $/kW_IT metric is therefore likely to remain in the $30,000-45,000 range for next-generation hardware. The economic benefit of Rubin accrues through cost-per-FLOP and cost-per-token improvements, not through reduced $/kW_IT.

GPU hardware dominates total cost

Across all analyses, GPU/server hardware represents 55-70% of total AI data center capex:

• McKinsey: $3.5T of $5.2T = 67%
• Thunder Said Energy: "over half" of $40K/kW
• SemiAnalysis: compute tray is the dominant cost component

This has a critical implication for orbital economics: moving data centers to space does not reduce the largest cost component (GPU hardware), which must still be manufactured on Earth and launched. The savings from orbital power and cooling must overcome the launch cost penalty applied to this hardware.