LLM VRAM Calculator: How Much GPU Memory You Need
Estimate the GPU memory of Llama, Qwen, DeepSeek, Mistral, Gemma, FLUX and SDXL for inference, LoRA or full fine-tuning, and see which GPUs hold it.
This calculator estimates how much GPU memory a model needs for inference, LoRA or full fine-tuning, from its size, weight precision, context length and batch size, and lists the GPUs that can hold it alone or in 2, 4 or 8 cards.
Estimate the VRAM of a model
Presets use the published configuration of each model (layers, key/value heads and hidden size). Choose Custom to enter the values of another model from its configuration file.
- Weights
- 140 GB
- KV cache
- 2.7 GB
- Overhead
- 14.3 GB
- Estimated total
- 157 GB
GPUs that can hold it
- NVIDIA B200 180GB1 × 180 GBRequest this configuration
- NVIDIA GB200 NVL1 × 186 GBRequest this configuration
- NVIDIA B300 288GB1 × 288 GBRequest this configuration
- NVIDIA GB300 NVL1 × 288 GBRequest this configuration
- NVIDIA A100 PCIe 80GB2 × 80 GBRequest this configuration
- NVIDIA A100 SXM 80GB2 × 80 GBRequest this configuration
- NVIDIA A800 PCIe 80GB2 × 80 GBRequest this configuration
- NVIDIA H100 PCIe 80GB2 × 80 GBRequest this configuration
- NVIDIA H100 SXM 80GB2 × 80 GBRequest this configuration
- NVIDIA H100 NVL 94GB2 × 94 GBRequest this configuration
- NVIDIA GH2002 × 96 GBRequest this configuration
- NVIDIA RTX PRO 6000 Blackwell 96GB2 × 96 GBRequest this configuration
Several GPUs assume the model is split across them (tensor or pipeline parallelism); interconnect and framework overhead are not counted. A capacity request does not reserve GPUs or confirm software compatibility or availability.
The page address keeps your inputs.
VRAM of popular open models
Inference estimates with an 8,192-token context, batch 1 and the default overhead (10 % for language models, 20 % for image pipelines), from the formulas below.
| Model | BF16 | FP8 | INT4 | Smallest single GPU (INT4) |
|---|---|---|---|---|
| Llama 3.2 3B | 8 GB | 5 GB | 3 GB | NVIDIA GeForce GTX 1060 6GB |
| Llama 3.1 8B | 19 GB | 10 GB | 6 GB | NVIDIA GeForce GTX 1060 6GB |
| Llama 3.3 70B | 158 GB | 81 GB | 42 GB | NVIDIA A40 48GB |
| Llama 3.1 405B | 898 GB | 451 GB | 228 GB | NVIDIA B300 288GB |
| Llama 3.2 1B | 3 GB | 2 GB | 1 GB | NVIDIA GeForce GTX 1060 6GB |
| Llama 3.2 11B Vision | 25 GB | 13 GB | 7 GB | NVIDIA GeForce GTX 1070 8GB |
| Llama 4 Scout (109B) | 242 GB | 122 GB | 62 GB | NVIDIA RTX PRO 5000 Blackwell 72GB |
| Llama 4 Maverick (400B) | 882 GB | 442 GB | 222 GB | NVIDIA B300 288GB |
| Qwen2.5 7B | 17 GB | 9 GB | 5 GB | NVIDIA GeForce GTX 1060 6GB |
| Qwen2.5 14B | 34 GB | 18 GB | 10 GB | NVIDIA GeForce RTX 3080 10GB |
| Qwen2.5 32B | 75 GB | 38 GB | 20 GB | NVIDIA A10 24GB |
| Qwen2.5 72B | 163 GB | 83 GB | 43 GB | NVIDIA A40 48GB |
| Qwen3 8B | 19 GB | 10 GB | 6 GB | NVIDIA GeForce GTX 1060 6GB |
| Qwen3 32B | 75 GB | 38 GB | 20 GB | NVIDIA A10 24GB |
| Qwen3 4B | 10 GB | 6 GB | 4 GB | NVIDIA GeForce GTX 1060 6GB |
| Qwen3 14B | 34 GB | 18 GB | 10 GB | NVIDIA GeForce RTX 3080 10GB |
| Qwen3 30B A3B | 68 GB | 34 GB | 18 GB | NVIDIA RTX 4000 Ada 20GB |
| Qwen3 235B A22B | 519 GB | 260 GB | 131 GB | NVIDIA H200 141GB |
| Qwen2.5 VL 7B | 19 GB | 10 GB | 5 GB | NVIDIA GeForce GTX 1060 6GB |
| Qwen2.5 VL 72B | 164 GB | 84 GB | 43 GB | NVIDIA A40 48GB |
| DeepSeek R1 Distill Qwen 7B | 17 GB | 9 GB | 5 GB | NVIDIA GeForce GTX 1060 6GB |
| DeepSeek R1 Distill Llama 8B | 19 GB | 10 GB | 6 GB | NVIDIA GeForce GTX 1060 6GB |
| DeepSeek R1 Distill Qwen 14B | 34 GB | 18 GB | 10 GB | NVIDIA GeForce RTX 3080 10GB |
| DeepSeek R1 Distill Qwen 32B | 75 GB | 38 GB | 20 GB | NVIDIA A10 24GB |
| DeepSeek R1 Distill Llama 70B | 158 GB | 81 GB | 42 GB | NVIDIA A40 48GB |
| DeepSeek V3 / R1 (671B) | 1,477 GB | 739 GB | 370 GB | Several GPUs |
| Kimi K2 (1T) | 2,258 GB | 1,129 GB | 565 GB | Several GPUs |
| GLM-4.5 (355B) | 784 GB | 394 GB | 199 GB | NVIDIA B300 288GB |
| GLM-4.5 Air (106B) | 235 GB | 118 GB | 60 GB | NVIDIA RTX PRO 5000 Blackwell 72GB |
| Mistral 7B | 17 GB | 9 GB | 5 GB | NVIDIA GeForce GTX 1060 6GB |
| Mistral Small 24B | 53 GB | 27 GB | 14 GB | NVIDIA A16 16GB |
| Mixtral 8x7B | 104 GB | 53 GB | 27 GB | NVIDIA GeForce RTX 5090 32GB |
| Mistral Large 123B | 274 GB | 139 GB | 71 GB | NVIDIA RTX PRO 5000 Blackwell 72GB |
| Mistral NeMo 12B | 28 GB | 15 GB | 8 GB | NVIDIA GeForce RTX 3080 10GB |
| Codestral 22B | 51 GB | 26 GB | 14 GB | NVIDIA A16 16GB |
| Mistral Small 3.1 24B | 54 GB | 28 GB | 15 GB | NVIDIA A16 16GB |
| Gemma 2 9B | 23 GB | 13 GB | 8 GB | NVIDIA GeForce RTX 3080 10GB |
| Gemma 2 27B | 63 GB | 33 GB | 18 GB | NVIDIA RTX 4000 Ada 20GB |
| Gemma 3 4B | 11 GB | 6 GB | 4 GB | NVIDIA GeForce GTX 1060 6GB |
| Gemma 3 12B | 30 GB | 17 GB | 10 GB | NVIDIA GeForce GTX 1080 Ti 11GB |
| Gemma 3 27B | 65 GB | 35 GB | 20 GB | NVIDIA RTX 4000 Ada 20GB |
| Phi-4 mini (3.8B) | 10 GB | 5 GB | 3 GB | NVIDIA GeForce GTX 1060 6GB |
| Phi-4 (14B) | 34 GB | 18 GB | 10 GB | NVIDIA GeForce RTX 3080 10GB |
| FLUX.1 [dev] (image) | 41 GB | 20 GB | 10 GB | NVIDIA GeForce GTX 1080 Ti 11GB |
| Stable Diffusion XL (image) | 8 GB | 4 GB | 2 GB | NVIDIA GeForce GTX 1060 6GB |
| Stable Diffusion 3.5 Large (image) | 33 GB | 17 GB | 8 GB | NVIDIA GeForce RTX 3080 10GB |
| Wan 2.1 1.3B (video) | 17 GB | 9 GB | 4 GB | NVIDIA GeForce GTX 1060 6GB |
| Wan 2.1 14B (video) | 48 GB | 24 GB | 12 GB | NVIDIA A16 16GB |
| CogVideoX 5B (video) | 25 GB | 13 GB | 6 GB | NVIDIA GeForce GTX 1070 8GB |
| HunyuanVideo (video) | 50 GB | 25 GB | 12 GB | NVIDIA A16 16GB |
Embed this calculator
Add the calculator to a blog post, course or documentation page for free. Paste this HTML where it should appear; it runs in the reader's browser and keeps the EverXCloud credit link.
<iframe src="https://everxcloud.ai/embed/llm-vram-calculator" title="LLM VRAM calculator by EverXCloud" width="100%" height="1100" style="border:0;max-width:960px" loading="lazy" referrerpolicy="strict-origin-when-cross-origin" sandbox="allow-scripts allow-same-origin allow-popups allow-popups-to-escape-sandbox"></iframe>
<p><a href="https://everxcloud.ai/en/tools/llm-vram-calculator">LLM VRAM calculator</a> by EverXCloud</p>How the estimate is calculated
- Weights (GB) = billions of parameters × bytes per parameter: 4 in FP32, 2 in BF16 or FP16, 1 in FP8 or INT8, 0.5 in INT4.
- KV cache (GB) = 2 × layers × KV dimension × context tokens × batch × 2 bytes ÷ 10⁹, with keys and values stored in BF16 or FP16. The KV dimension is the number of key/value heads × the size of one head.
- Inference total (GB) = (weights + KV cache) × (1 + overhead %), the overhead covering activations, CUDA context and runtime buffers.
- LoRA fine-tune: frozen weights at the chosen precision (QLoRA when INT4) + adapters trained on 1 % of the parameters (a cautious upper bound) at 16 bytes each + activations.
- Full fine-tune: 16 bytes per parameter with AdamW in BF16 mixed precision (2 for weights, 2 for gradients, 4 for the FP32 master copy, 8 for the two Adam moments) + activations.
- Training activations (GB) = sequence tokens × batch × hidden size × (2 × layers + 34) bytes ÷ 10⁹, with gradient checkpointing and fused attention (Korthikanti et al., 2022), plus the logits and their gradient: sequence tokens × batch × vocabulary × 8 bytes (FP32). SwiGLU MLPs keep slightly more than the 34 bytes counted; the overhead % applies to the training total too.
- Image models (FLUX.1, SDXL) count the whole pipeline (denoiser, text encoders and VAE) for one 1024 × 1024 image, with 20 % overhead and no KV cache.
- A model fits on n GPUs when n × the memory of one GPU ≥ the total, assuming it is split across them.
Frequently asked questions
- How is the VRAM of an LLM calculated?
- Weights take billions of parameters × bytes per parameter (2 in BF16, 1 in FP8 or INT8, 0.5 in INT4). The KV cache adds 2 × layers × KV dimension × context tokens × batch × 2 bytes. A margin covers activations and runtime buffers.
- How much VRAM does a 70B model need?
- Its weights take 140 GB in BF16, 70 GB in FP8 or INT8 and 35 GB in INT4, before the KV cache and runtime overhead. In BF16 it needs several GPUs or a single GPU with more than 140 GB of memory.
- How much VRAM do you need to run Llama 3.3 70B?
- About 158 GB in BF16, 81 GB in FP8 and 42 GB in INT4 with an 8,192-token context, batch 1 and 10 % overhead. That is 2 GPUs of 80 GB in BF16, 2 GPUs of 80 GB in FP8, and one 48 GB GPU in INT4.
- Can an RTX 4090 run DeepSeek R1 Distill Qwen 32B?
- Yes, in INT4: the 32B distill needs about 20 GB in INT4 with an 8,192-token context, within the 24 GB of an RTX 4090. In BF16 it needs about 75 GB, so 4 GPUs of 24 GB or a single GPU with more memory. Longer contexts or larger batches grow the KV cache.
- How much VRAM does fine-tuning an LLM need?
- Full fine-tuning with AdamW takes about 16 bytes per parameter before activations: about 145 GB for Llama 3.1 8B with 2,048-token sequences. LoRA keeps the base model frozen: about 22 GB with BF16 weights, and about 9 GB with QLoRA (INT4 base).
- How much VRAM does FLUX.1 [dev] need?
- The full FLUX.1 [dev] pipeline (transformer, T5-XXL and CLIP text encoders, VAE) needs about 41 GB in BF16 and 20 GB in FP8 for one 1024 × 1024 image. Offloading the text encoder once the prompt is encoded brings BF16 within reach of a 32 GB GPU.
- Is the estimate exact?
- No. Real usage depends on the model architecture, the inference engine, quantization metadata and memory fragmentation. Treat the result as a planning estimate and test on the target GPU before committing to a long rental.
Explore the live market
Every offer on the live market comes from a reviewed supplier and is sold by EverXCloud Inc.