GPUs for training large language models
Training a model from scratch, or fully fine-tuning every weight, needs far more memory than running it: weights, gradients and optimizer states all live on the GPU. Data center GPUs with HBM memory, fast tensor cores and a GPU-to-GPU interconnect are the practical choice.
GPUs that fit LLM training
Swipe sideways to see every column.
| GPU | Memory | Memory type | Bandwidth | BF16 tensor | Full training, 1 GPU | Full training, 8 GPUs | Price/h |
|---|---|---|---|---|---|---|---|
| B300Blackwell Ultra | 288 GB | HBM3e | Not published | Not published | ≈ 10B | ≈ 110B | from $8.99 |
| B200Blackwell | 180 GB | HBM3e | Not published | Not published | ≈ 9B | ≈ 70B | from $7.99 |
| MI325XCDNA 3 | 256 GB | HBM3e | 6,000 GB/s | 1,307 TFLOPS | ≈ 10B | ≈ 100B | from $4.82 |
| MI300XCDNA 3 | 192 GB | HBM3 | 5,300 GB/s | 1,307 TFLOPS | ≈ 9B | ≈ 75B | from $4.16 |
| H200Hopper | 141 GB | HBM3e | 4,800 GB/s | 989 TFLOPS | ≈ 7B | ≈ 55B | from $4.19 |
| H100 NVLHopper | 94 GB | HBM3 | 3,900 GB/s | 835 TFLOPS | ≈ 4B | ≈ 35B | from $3.19 |
| H100 SXMHopper | 80 GB | HBM3 | 3,350 GB/s | 989 TFLOPS | ≈ 4B | ≈ 30B | from $2.88 |
| A100 SXMAmpere | 80 GB | HBM2e | 2,039 GB/s | 312 TFLOPS | ≈ 4B | ≈ 30B | from $1.79 |
| A100 PCIeAmpere | 80 GB | HBM2e | 1,935 GB/s | 312 TFLOPS | ≈ 4B | ≈ 30B | from $1.79 |
Manufacturer specifications, per GPU; tensor throughput without sparsity. Model sizes are cautious estimates in billions of parameters, counting 80% of memory; context length, batch size and framework change the real limit. Prices are the best live offer for one GPU per hour, excluding tax.
All GPU modelsWhat matters for LLM training
Memory sets the model size
Mixed-precision training with the Adam optimizer needs about 16 bytes per parameter before activations. One 80 GB GPU therefore trains models of a few billion parameters; larger models are split across 8 GPUs or more with sharded data parallelism (FSDP, ZeRO) or tensor parallelism.
Tensor throughput and precision
Training runs mostly in BF16, and newer GPUs add FP8. Dense tensor throughput is what turns into tokens per second: compare the BF16 column, and keep in mind that published figures with sparsity are twice the dense value.
Interconnect for multi-GPU jobs
Once a model is split across GPUs, they exchange gradients at every step. SXM boards with NVLink, or AMD Infinity Fabric, keep that exchange fast; PCIe cards suit smaller jobs and data parallel runs where each GPU holds a full copy of the model.
Compare GPUs for LLM training
GPUs for training large language models: frequently asked questions
How much GPU memory do I need to train an LLM?
As a cautious rule, about 16 bytes per parameter for weights, gradients and Adam states in mixed precision, plus activations. A 7-billion-parameter model needs more than 100 GB in total, so it is trained on several 80 GB GPUs, or on fewer GPUs with sharding and activation checkpointing.
H100 or A100 for training?
Both have 80 GB of memory. The H100 SXM publishes about three times the dense BF16 tensor throughput of the A100 SXM and adds FP8, with higher memory bandwidth; the A100 remains a sound choice for jobs that are not bound by compute.
Can I train on GeForce cards?
For small models and experiments, yes. For larger runs, 24 to 32 GB per card and the lack of a fast GPU-to-GPU interconnect limit how far a job can scale; data center GPUs are listed first on this page for that reason.
How do I rent a GPU for LLM training on EverXCloud?
Create an account, top up your prepaid balance in euros, US dollars or pounds, then buy GPU hours on the spot exchange at the best offer, or place a request at the price you are willing to pay. Rentals are billed by the hour. Prices are shown excluding tax and EverXCloud is the merchant of record for every rental.
Plan your GPU rental
- GPU rental: how to rent a cloud GPU by the hour
- How to rent your first GPU
- How to choose a GPU for LLM inference
- How to estimate a GPU rental session
- H100 price per hour: what sets the rate
- How a GPU spot exchange works
- H200 price per hour: cloud pricing explained
- B200 price per hour: B200, B300 and GB200
- A100 price per hour: rent an NVIDIA A100
- RTX 4090 price per hour: rental guide
- RTX 5090 price per hour: rental guide
- L40S price per hour: rental guide
- MI300X price per hour: rental guide
- How much VRAM do you need for an LLM?
- Best GPU for Stable Diffusion and Flux
- Cheapest way to rent a GPU for AI
- Best GPU for scientific computing (FP64)
Check the workload, complete configuration and billing conditions before choosing a live offer.
Be ready on launch day
Create your account now: you can top up your balance and rent the GPU that fits your workload as soon as the market opens.
Have GPU servers?
Businesses can join the EverXCloud Network and sell hours of their GPU servers on the spot exchange.
Other workloads
- GPUs for fine-tuning open models
LoRA and QLoRA adapters on a single GPU, from 24 GB up.
- GPUs for LLM inference
Serve open models: what fits in memory, and how fast tokens come.
- GPUs for image and video generation
Diffusion models for images and video, from 24 GB cards to 80 GB GPUs.
- GPUs for 3D rendering and visual effects
Ray-traced rendering, visual effects and real-time engines.
- GPUs for scientific computing and simulation
Simulation and HPC codes on data center GPUs with HBM memory.