GPUs for fine-tuning open models
Parameter-efficient fine-tuning trains small adapters on top of a frozen base model. Memory needs drop sharply, so a single GPU often does the job: the base model only has to fit, in 16-bit with LoRA or in 4-bit with QLoRA, with room left for activations.
GPUs that fit fine-tuning
Swipe sideways to see every column.
| GPU | Memory | Bandwidth | QLoRA, base model | LoRA 16-bit, base model | Price/h |
|---|---|---|---|---|---|
| RTX 4090Ada Lovelace | 24 GB | 1,008 GB/s | ≈ 15B | ≈ 7B | from $0.33 |
| RTX 5090Blackwell | 32 GB | 1,792 GB/s | ≈ 25B | ≈ 10B | from $0.69 |
| RTX A6000Ampere | 48 GB | 768 GB/s | ≈ 35B | ≈ 15B | from $0.33 |
| L40SAda Lovelace | 48 GB | 864 GB/s | ≈ 35B | ≈ 15B | from $0.79 |
| RTX 6000 AdaAda Lovelace | 48 GB | 960 GB/s | ≈ 35B | ≈ 15B | from $0.74 |
| A100 SXMAmpere | 80 GB | 2,039 GB/s | ≈ 60B | ≈ 25B | from $1.79 |
| H100 SXMHopper | 80 GB | 3,350 GB/s | ≈ 60B | ≈ 25B | from $2.88 |
| RTX PRO 6000 BlackwellBlackwell | 96 GB | 1,792 GB/s | ≈ 75B | ≈ 30B | from $1.69 |
| H200Hopper | 141 GB | 4,800 GB/s | ≈ 110B | ≈ 45B | from $4.19 |
Manufacturer specifications, per GPU; tensor throughput without sparsity. Model sizes are cautious estimates in billions of parameters, counting 80% of memory; context length, batch size and framework change the real limit. Prices are the best live offer for one GPU per hour, excluding tax.
All GPU modelsWhat matters for fine-tuning
LoRA or QLoRA
LoRA keeps the base model in 16-bit and trains low-rank adapters: about 2 bytes per parameter for the frozen weights. QLoRA quantizes the base model to 4-bit, cutting that to well under 1 byte per parameter, at the cost of slower steps. The table estimates the largest base model for each, with headroom.
Sequence length and batch size
Activations grow with sequence length and batch size. Long-context fine-tuning needs more headroom than the estimates assume; gradient checkpointing and smaller micro-batches trade speed for memory.
One GPU first
A 24 GB or 32 GB card is enough to fine-tune many 7 to 14 billion parameter models with QLoRA. Move to 48 GB or 80 GB GPUs for 16-bit LoRA on larger models, or to several GPUs for full fine-tuning.
Compare GPUs for fine-tuning
GPUs for fine-tuning open models: frequently asked questions
Which GPU do I need to fine-tune a 7B model?
With QLoRA, a 24 GB GPU such as the RTX 4090 is usually enough for a 7 or 8 billion parameter model at moderate sequence lengths. With 16-bit LoRA, a 48 GB GPU leaves comfortable headroom.
Is QLoRA slower than LoRA?
Usually, yes: the 4-bit weights are dequantized at every step. QLoRA trades some speed for a much smaller memory footprint, which lets a cheaper GPU or a larger base model fit.
When do I need full fine-tuning instead?
When adapters are not enough for the change you want, every weight is trained and memory needs return to those of training, about 16 bytes per parameter. See the LLM training page for GPUs that fit.
How do I rent a GPU for fine-tuning on EverXCloud?
Create an account, top up your prepaid balance in euros, US dollars or pounds, then buy GPU hours on the spot exchange at the best offer, or place a request at the price you are willing to pay. Rentals are billed by the hour. Prices are shown excluding tax and EverXCloud is the merchant of record for every rental.
Plan your GPU rental
- GPU rental: how to rent a cloud GPU by the hour
- How to rent your first GPU
- How to choose a GPU for LLM inference
- How to estimate a GPU rental session
- H100 price per hour: what sets the rate
- How a GPU spot exchange works
- H200 price per hour: cloud pricing explained
- B200 price per hour: B200, B300 and GB200
- A100 price per hour: rent an NVIDIA A100
- RTX 4090 price per hour: rental guide
- RTX 5090 price per hour: rental guide
- L40S price per hour: rental guide
- MI300X price per hour: rental guide
- How much VRAM do you need for an LLM?
- Best GPU for Stable Diffusion and Flux
- Cheapest way to rent a GPU for AI
- Best GPU for scientific computing (FP64)
Check the workload, complete configuration and billing conditions before choosing a live offer.
Be ready on launch day
Create your account now: you can top up your balance and rent the GPU that fits your workload as soon as the market opens.
Have GPU servers?
Businesses can join the EverXCloud Network and sell hours of their GPU servers on the spot exchange.
Other workloads
- GPUs for training large language models
Pre-training and full fine-tuning on data center GPUs with HBM memory.
- GPUs for LLM inference
Serve open models: what fits in memory, and how fast tokens come.
- GPUs for image and video generation
Diffusion models for images and video, from 24 GB cards to 80 GB GPUs.
- GPUs for 3D rendering and visual effects
Ray-traced rendering, visual effects and real-time engines.
- GPUs for scientific computing and simulation
Simulation and HPC codes on data center GPUs with HBM memory.