Total Cost / GPU / hr$1.553/hr
Capital / GPU / hr$1.115/hr71.8% of total
Operating / GPU / hr$0.438/hr28.2% of total
Total CapEx$349K$44K per GPU
TCO / PFLOP$0.785per hour
TCO / Training PFLOP$2.071per hour
Capital vs Operating Split
Cost composition for H100 SXM (InfiniBand)
Capital71.8%
Operating28.2%
Annual Operating Costs
$30,710 per year
Capital Cost Breakdown
Upfront investment — 8-GPU Server
| Component | Amount | % of Total |
|---|---|---|
| Server / Compute | $244,933 | 70.2% |
| Networking | $76,106 | 21.8% |
| Storage | $11,732 | 3.4% |
| Software & Licenses | $2,867 | 0.8% |
| Service | $8,000 | 2.3% |
| Installation | $5,500 | 1.6% |
| Total CapEx | $349,138 | 100.0% |
GPU Performance Context
H100 SXM specifications and efficiency metrics
FP8 TFLOPS1,978
Effective Training TFLOPS750
Inference (tok/s)90
Memory BW (TB/s)3.0
Memory (GB)80
Power (kW)10.9
GPUs per Server8
Utilization80.0%