Glossary3 sections1 min read
What is LoRA and QLoRA?
LoRA (Low-Rank Adaptation) fine-tunes a model by training small low-rank adapter matrices while the original weights stay frozen. QLoRA does the same on top of a base model quantized to 4 bits.
Why it matters when renting a GPU
LoRA avoids storing gradients and optimizer states for every weight, so fine-tuning fits on far smaller GPUs than full fine-tuning.
QLoRA goes further by keeping the frozen base model in 4 bits.
Worked example
With QLoRA, the frozen weights of a 70B model take about 70 × 0.5 = 35 GB, which fits within the 80 GB of one H100 with room left for adapters and activations; full FP16 weights alone would need 140 GB.
Frequently asked questions
- Can I fine-tune on an RTX 4090?
- With LoRA or QLoRA, models up to roughly the size whose quantized weights fit in 24 GB plus working memory are practical.