H100 SXM vs A100 SXM
Manufacturer specifications of the H100 SXM and the A100 SXM, side by side, per GPU. A value the manufacturer does not publish for the exact model is marked as such, never estimated.
NVIDIA H100 SXM 80GB
Hopper
- GPU memory
- 80 GB
- Memory bandwidth
- 3,350 GB/s
Live price on EverXCloud · from $2.88 per GPU-hour
Suited workloads
NVIDIA A100 SXM 80GB
Ampere
- GPU memory
- 80 GB
- Memory bandwidth
- 2,039 GB/s
Live price on EverXCloud · from $1.79 per GPU-hour
Suited workloads
Key differences
- Both have 80 GB of GPU memory.
- The H100 SXM has 1.6× the memory bandwidth of the A100 SXM (3,350 vs 2,039 GB/s), which usually sets the pace of LLM token generation.
- The H100 SXM has 3.2× the dense BF16 tensor throughput of the A100 SXM (989 vs 312 TFLOPS), the precision of most training runs.
Specifications
| Specification | H100 SXM | A100 SXM |
|---|---|---|
| GPU memory | 80 GB | 80 GB |
| Memory type | HBM3 | HBM2e |
| Memory bandwidth | 3,350 GB/s (higher) | 2,039 GB/s |
| FP16 tensor | 989 TFLOPS (higher) | 312 TFLOPS |
| BF16 tensor | 989 TFLOPS (higher) | 312 TFLOPS |
| FP8 tensor | 1,979 TFLOPS | Not published |
| Max power | 700 W | 400 W |
| Architecture | Hopper | Ampere |
| Interconnect | NVLink 4 (900 GB/s) · PCIe 5.0 | NVLink 3 (600 GB/s) · PCIe 4.0 |
The higher value of each measured row is shown in bold. Manufacturer specifications, per GPU. Tensor throughput is shown without sparsity. Values the manufacturer does not publish for this exact model are left out.
What fits in memory, on one GPU
| Workload | H100 SXM | A100 SXM |
|---|---|---|
| LLM inference, 16-bit | ≈ 30B parameters | ≈ 30B parameters |
| LLM inference, 4-bit | ≈ 100B parameters | ≈ 100B parameters |
| Full training, mixed precision | ≈ 4B parameters | ≈ 4B parameters |
Cautious estimates counting 80% of memory for model weights; context length, batch size and framework change the real limit. Several GPUs together hold larger models.
H100 SXM vs A100 SXM: frequently asked questions
Which has more memory, the H100 SXM or the A100 SXM?
Both have 80 GB of GPU memory.
Which is faster for LLM inference, the H100 SXM or the A100 SXM?
Generating tokens is usually bound by memory bandwidth, and the H100 SXM has 1.6× the bandwidth of the A100 SXM. Memory size then decides which models fit. Measure your own model before a large run.
Can I rent the H100 SXM and the A100 SXM by the hour?
Yes, when sellers list them on the EverXCloud spot exchange: you buy GPU hours at the live price from a prepaid balance in euros, US dollars or pounds. Prices are shown excluding tax and EverXCloud is the merchant of record for every rental.
Be ready on launch day
Create your account now: you can top up your balance and rent the GPU that fits your workload as soon as the market opens.
Have GPU servers?
Businesses can join the EverXCloud Network and sell hours of their GPU servers on the spot exchange.