GPU Cost per kW_IT

What is the cost per kW_IT for current-generation AI accelerator hardware?

Answer

The cost per kW_IT for current-generation AI accelerator hardware (GB200 NVL72-class rack equipment) ranges from $25,000 to $40,000/kW_IT, with a central estimate of $32,500/kW_IT. Here, the metric refers to rack-equipment cost: compute trays, NVLink switches, CDU, in-rack power distribution, and rack-adjacent networking/storage. It excludes building/site infrastructure, power plant capex, and shell-and-core construction.

CDU cost boundary note: The $32,500/kW_IT figure includes the CDU (~$50K per rack, or ~$417/kW_IT) as part of rack equipment, since the CDU ships with the NVL72 rack and is integral to its operation. The terrestrial-infrastructure-cost page's cooling line item ($1,500-$4,000/kW_IT) covers facility-level cooling infrastructure -- cooling towers/chillers, secondary loop piping, heat rejection to the environment -- that sits outside the rack boundary. There is no double-count: the CDU transfers heat from chips to the facility cooling loop; the facility cooling infrastructure rejects that heat to the atmosphere.

At hyperscaler volume pricing (~$3.1M per rack at 120 kW), the rack-equipment cost is ~$25,800/kW_IT. At all-in rack cost including networking and storage (~$3.9M), the figure rises to ~$32,500/kW_IT. For smaller buyers or fully provisioned rack-equipment stacks with higher vendor margin, sparing, and integration overhead, costs can reach ~$40,000/kW_IT without adding facility infrastructure.

Analysis

Central estimate rationale

The central estimate of $32,500/kW_IT uses the all-in rack cost ($3.9M) divided by IT load (120 kW). This is appropriate because:

1. It includes compute hardware, NVLink switches, CDU, power distribution, and networking -- all components that must be launched for an orbital deployment comparison.
2. It excludes data center shell and site infrastructure, which would be replaced by spacecraft bus and solar arrays in an orbital scenario.
3. It aligns with McKinsey's IT equipment estimate of ~$26,400/kW_IT (the difference being McKinsey's figure is a fleet average including cheaper non-AI servers and includes storage/networking).

Optimistic and conservative bounds

• $25,000/kW_IT (optimistic): Hyperscaler volume pricing for the rack hardware itself, excluding rack-adjacent networking and storage. Achievable by the largest buyers with long-term supply agreements.
• $40,000/kW_IT (conservative): High-end rack-equipment pricing for smaller buyers or early deployments, including networking/storage, sparing, integration overhead, and higher vendor margin at the rack level. This is an upper guardrail for compute-stack cost, not a facility-infrastructure number.

Will Rubin change $/kW_IT?

Vera Rubin NVL72 delivers substantially more compute per rack (up to 3.5x dense FP4 per GPU per SemiAnalysis) at a higher power envelope (180-220 kW TDP, up from 120-140 kW for GB200). The rack price is expected to increase roughly proportionally with compute content. SemiAnalysis data shows compute tray costs rising from ~$170K (H100) to ~$264K (B200) to ~$344K (B300), a trajectory that suggests VR NVL72 rack costs will be $4-5M+.

The $/kW_IT metric is therefore likely to remain in the $30,000-45,000 range for next-generation hardware. The economic benefit of Rubin accrues through cost-per-FLOP and cost-per-token improvements, not through reduced $/kW_IT.

GPU hardware dominates total cost

Across all analyses, GPU/server hardware represents 55-70% of total AI data center capex:

• McKinsey: $3.3T IT equipment of $5.2T = 63% (or $3.1T tech developers = 60%)
• Thunder Said Energy: "over half" of $40K/kW
• SemiAnalysis: compute tray is the dominant cost component

This has a critical implication for orbital economics: moving data centers to space does not reduce the largest cost component (GPU hardware), which must still be manufactured on Earth and launched. The savings from orbital power and cooling must overcome the launch cost penalty applied to this hardware.

Evidence

GB200 NVL72 rack economics

1. GB200 NVL72 rack ~$3.1M (hyperscaler price), ~$3.9M all-in. TCO is ~1.6x higher than H100 per GPU. Rack power draw of 120 kW per NVIDIA specifications. (SemiAnalysis, 2025) -- semianalysis-gb200-tco

2. NVIDIA's DGX GB200 documentation specifies the system at 120 kW, providing the baseline power figure used in the $/kW_IT conversion. (NVIDIA, 2025) -- nvidia-gb200-specs

Derived $/kW_IT calculations

Comparison with industry benchmarks

3. Projects $6.7T total data center capex by 2030 ($5.2T for AI workloads, $1.5T for non-AI). The $5.2T AI figure assumes 125 GW incremental AI capacity ("continued momentum" scenario; range: $3.7T at 78 GW constrained to $7.9T at 205 GW accelerated). Exhibit 2 breaks down the $5.2T: DC infrastructure $1.6T, IT equipment $3.3T, power $0.3T. By investor archetype: 60% ($3.1T) to technology developers (chips and computing hardware), 25% ($1.3T) to energizers (power, cooling, electrical), 15% ($0.8T) to builders (land, site development). This implies ~$26,400/kW_IT for IT equipment ($3.3T / 125 GW) and ~$41,600/kW_IT all-in ($5.2T / 125 GW). (McKinsey, April 2025) -- mckinsey-cost-of-compute

4. Standard data centers cost ~$10,000/kW; AI-heavy computation reaches up to $40,000/kW, with "over half" being GPU hardware (~$20,000+/kW for servers). (Thunder Said Energy) -- thunder-said-dc-economics

5. Thunder Said Energy estimates AI-heavy data centers can reach roughly $40,000/kW all-in (including facility infrastructure), with GPU/server hardware accounting for more than half of that total. This all-in figure serves as a cross-check for the conservative bound, though the page's $40,000/kW_IT conservative estimate refers to rack-equipment cost only. (Thunder Said Energy) -- thunder-said-dc-economics

SemiAnalysis TCO component breakdown

6. SemiAnalysis InferenceX v2 analysis of compute tray content costs: H100 SXM ~$170K, B200 ~$264K, B300 ~$344K, MI300X ~$138K, MI355X ~$197K. The trend shows GPU content cost increasing with each generation, driven by higher GPU pricing and NVIDIA's premium margins. (SemiAnalysis, Feb 2026) -- semianalysis-inferencex-v2

Next-generation outlook: Vera Rubin

7. Vera Rubin NVL72 maintains the rack-scale Oberon architecture with 72 Rubin GPUs, 36 Vera CPUs, and 36 NVLink 6 Switch ASICs. System TDP increases to 180-220 kW per rack (up from 120-140 kW for GB200 NVL72), reflecting higher compute density (SemiAnalysis, Feb 2026). NVIDIA claims up to 4x better training performance, up to 10x better inference performance per watt, and one-tenth the token cost relative to Blackwell (NVIDIA vera-rubin-nvl72-nvidia). Cost-per-FLOP drops dramatically, but cost-per-kW_IT may increase due to higher silicon content per watt. -- semianalysis-vera-rubin

8. Vera Rubin increases rack power density substantially versus GB200 while preserving the rack-scale architecture, which means cost-per-FLOP should fall faster than cost-per-kW_IT. Detailed 2026 production-capacity and cooling-cost claims are omitted here because they are not yet backed by sources in sources.md. -- semianalysis-vera-rubin