Skip to content

LLM VRAM Calculator: How Much GPU Memory You Need

Estimate the GPU memory of Llama, Qwen, DeepSeek, Mistral, Gemma, FLUX and SDXL for inference, LoRA or full fine-tuning, and see which GPUs hold it.

This calculator estimates how much GPU memory a model needs for inference, LoRA or full fine-tuning, from its size, weight precision, context length and batch size, and lists the GPUs that can hold it alone or in 2, 4 or 8 cards.

Estimate the VRAM of a model

Presets use the published configuration of each model (layers, key/value heads and hidden size). Choose Custom to enter the values of another model from its configuration file.

Weights
140 GB
KV cache
2.7 GB
Overhead
14.3 GB
Estimated total
157 GB

GPUs that can hold it

Several GPUs assume the model is split across them (tensor or pipeline parallelism); interconnect and framework overhead are not counted. A capacity request does not reserve GPUs or confirm software compatibility or availability.

The page address keeps your inputs.

VRAM of popular open models

Inference estimates with an 8,192-token context, batch 1 and the default overhead (10 % for language models, 20 % for image pipelines), from the formulas below.

Estimated inference VRAM by model and weight precision
Llama 3.2 3B8 GB5 GB3 GBNVIDIA GeForce GTX 1060 6GB
Llama 3.1 8B19 GB10 GB6 GBNVIDIA GeForce GTX 1060 6GB
Llama 3.3 70B158 GB81 GB42 GBNVIDIA A40 48GB
Llama 3.1 405B898 GB451 GB228 GBNVIDIA B300 288GB
Llama 3.2 1B3 GB2 GB1 GBNVIDIA GeForce GTX 1060 6GB
Llama 3.2 11B Vision25 GB13 GB7 GBNVIDIA GeForce GTX 1070 8GB
Llama 4 Scout (109B)242 GB122 GB62 GBNVIDIA RTX PRO 5000 Blackwell 72GB
Llama 4 Maverick (400B)882 GB442 GB222 GBNVIDIA B300 288GB
Qwen2.5 7B17 GB9 GB5 GBNVIDIA GeForce GTX 1060 6GB
Qwen2.5 14B34 GB18 GB10 GBNVIDIA GeForce RTX 3080 10GB
Qwen2.5 32B75 GB38 GB20 GBNVIDIA A10 24GB
Qwen2.5 72B163 GB83 GB43 GBNVIDIA A40 48GB
Qwen3 8B19 GB10 GB6 GBNVIDIA GeForce GTX 1060 6GB
Qwen3 32B75 GB38 GB20 GBNVIDIA A10 24GB
Qwen3 4B10 GB6 GB4 GBNVIDIA GeForce GTX 1060 6GB
Qwen3 14B34 GB18 GB10 GBNVIDIA GeForce RTX 3080 10GB
Qwen3 30B A3B68 GB34 GB18 GBNVIDIA RTX 4000 Ada 20GB
Qwen3 235B A22B519 GB260 GB131 GBNVIDIA H200 141GB
Qwen2.5 VL 7B19 GB10 GB5 GBNVIDIA GeForce GTX 1060 6GB
Qwen2.5 VL 72B164 GB84 GB43 GBNVIDIA A40 48GB
DeepSeek R1 Distill Qwen 7B17 GB9 GB5 GBNVIDIA GeForce GTX 1060 6GB
DeepSeek R1 Distill Llama 8B19 GB10 GB6 GBNVIDIA GeForce GTX 1060 6GB
DeepSeek R1 Distill Qwen 14B34 GB18 GB10 GBNVIDIA GeForce RTX 3080 10GB
DeepSeek R1 Distill Qwen 32B75 GB38 GB20 GBNVIDIA A10 24GB
DeepSeek R1 Distill Llama 70B158 GB81 GB42 GBNVIDIA A40 48GB
DeepSeek V3 / R1 (671B)1,477 GB739 GB370 GBSeveral GPUs
Kimi K2 (1T)2,258 GB1,129 GB565 GBSeveral GPUs
GLM-4.5 (355B)784 GB394 GB199 GBNVIDIA B300 288GB
GLM-4.5 Air (106B)235 GB118 GB60 GBNVIDIA RTX PRO 5000 Blackwell 72GB
Mistral 7B17 GB9 GB5 GBNVIDIA GeForce GTX 1060 6GB
Mistral Small 24B53 GB27 GB14 GBNVIDIA A16 16GB
Mixtral 8x7B104 GB53 GB27 GBNVIDIA GeForce RTX 5090 32GB
Mistral Large 123B274 GB139 GB71 GBNVIDIA RTX PRO 5000 Blackwell 72GB
Mistral NeMo 12B28 GB15 GB8 GBNVIDIA GeForce RTX 3080 10GB
Codestral 22B51 GB26 GB14 GBNVIDIA A16 16GB
Mistral Small 3.1 24B54 GB28 GB15 GBNVIDIA A16 16GB
Gemma 2 9B23 GB13 GB8 GBNVIDIA GeForce RTX 3080 10GB
Gemma 2 27B63 GB33 GB18 GBNVIDIA RTX 4000 Ada 20GB
Gemma 3 4B11 GB6 GB4 GBNVIDIA GeForce GTX 1060 6GB
Gemma 3 12B30 GB17 GB10 GBNVIDIA GeForce GTX 1080 Ti 11GB
Gemma 3 27B65 GB35 GB20 GBNVIDIA RTX 4000 Ada 20GB
Phi-4 mini (3.8B)10 GB5 GB3 GBNVIDIA GeForce GTX 1060 6GB
Phi-4 (14B)34 GB18 GB10 GBNVIDIA GeForce RTX 3080 10GB
FLUX.1 [dev] (image)41 GB20 GB10 GBNVIDIA GeForce GTX 1080 Ti 11GB
Stable Diffusion XL (image)8 GB4 GB2 GBNVIDIA GeForce GTX 1060 6GB
Stable Diffusion 3.5 Large (image)33 GB17 GB8 GBNVIDIA GeForce RTX 3080 10GB
Wan 2.1 1.3B (video)17 GB9 GB4 GBNVIDIA GeForce GTX 1060 6GB
Wan 2.1 14B (video)48 GB24 GB12 GBNVIDIA A16 16GB
CogVideoX 5B (video)25 GB13 GB6 GBNVIDIA GeForce GTX 1070 8GB
HunyuanVideo (video)50 GB25 GB12 GBNVIDIA A16 16GB

Embed this calculator

Add the calculator to a blog post, course or documentation page for free. Paste this HTML where it should appear; it runs in the reader's browser and keeps the EverXCloud credit link.

<iframe src="https://everxcloud.ai/embed/llm-vram-calculator" title="LLM VRAM calculator by EverXCloud" width="100%" height="1100" style="border:0;max-width:960px" loading="lazy" referrerpolicy="strict-origin-when-cross-origin" sandbox="allow-scripts allow-same-origin allow-popups allow-popups-to-escape-sandbox"></iframe>
<p><a href="https://everxcloud.ai/en/tools/llm-vram-calculator">LLM VRAM calculator</a> by EverXCloud</p>

How the estimate is calculated

  • Weights (GB) = billions of parameters × bytes per parameter: 4 in FP32, 2 in BF16 or FP16, 1 in FP8 or INT8, 0.5 in INT4.
  • KV cache (GB) = 2 × layers × KV dimension × context tokens × batch × 2 bytes ÷ 10⁹, with keys and values stored in BF16 or FP16. The KV dimension is the number of key/value heads × the size of one head.
  • Inference total (GB) = (weights + KV cache) × (1 + overhead %), the overhead covering activations, CUDA context and runtime buffers.
  • LoRA fine-tune: frozen weights at the chosen precision (QLoRA when INT4) + adapters trained on 1 % of the parameters (a cautious upper bound) at 16 bytes each + activations.
  • Full fine-tune: 16 bytes per parameter with AdamW in BF16 mixed precision (2 for weights, 2 for gradients, 4 for the FP32 master copy, 8 for the two Adam moments) + activations.
  • Training activations (GB) = sequence tokens × batch × hidden size × (2 × layers + 34) bytes ÷ 10⁹, with gradient checkpointing and fused attention (Korthikanti et al., 2022), plus the logits and their gradient: sequence tokens × batch × vocabulary × 8 bytes (FP32). SwiGLU MLPs keep slightly more than the 34 bytes counted; the overhead % applies to the training total too.
  • Image models (FLUX.1, SDXL) count the whole pipeline (denoiser, text encoders and VAE) for one 1024 × 1024 image, with 20 % overhead and no KV cache.
  • A model fits on n GPUs when n × the memory of one GPU ≥ the total, assuming it is split across them.

Frequently asked questions

How is the VRAM of an LLM calculated?
Weights take billions of parameters × bytes per parameter (2 in BF16, 1 in FP8 or INT8, 0.5 in INT4). The KV cache adds 2 × layers × KV dimension × context tokens × batch × 2 bytes. A margin covers activations and runtime buffers.
How much VRAM does a 70B model need?
Its weights take 140 GB in BF16, 70 GB in FP8 or INT8 and 35 GB in INT4, before the KV cache and runtime overhead. In BF16 it needs several GPUs or a single GPU with more than 140 GB of memory.
How much VRAM do you need to run Llama 3.3 70B?
About 158 GB in BF16, 81 GB in FP8 and 42 GB in INT4 with an 8,192-token context, batch 1 and 10 % overhead. That is 2 GPUs of 80 GB in BF16, 2 GPUs of 80 GB in FP8, and one 48 GB GPU in INT4.
Can an RTX 4090 run DeepSeek R1 Distill Qwen 32B?
Yes, in INT4: the 32B distill needs about 20 GB in INT4 with an 8,192-token context, within the 24 GB of an RTX 4090. In BF16 it needs about 75 GB, so 4 GPUs of 24 GB or a single GPU with more memory. Longer contexts or larger batches grow the KV cache.
How much VRAM does fine-tuning an LLM need?
Full fine-tuning with AdamW takes about 16 bytes per parameter before activations: about 145 GB for Llama 3.1 8B with 2,048-token sequences. LoRA keeps the base model frozen: about 22 GB with BF16 weights, and about 9 GB with QLoRA (INT4 base).
How much VRAM does FLUX.1 [dev] need?
The full FLUX.1 [dev] pipeline (transformer, T5-XXL and CLIP text encoders, VAE) needs about 41 GB in BF16 and 20 GB in FP8 for one 1024 × 1024 image. Offloading the text encoder once the prompt is encoded brings BF16 within reach of a 32 GB GPU.
Is the estimate exact?
No. Real usage depends on the model architecture, the inference engine, quantization metadata and memory fragmentation. Treat the result as a planning estimate and test on the target GPU before committing to a long rental.

Explore the live market

Every offer on the live market comes from a reviewed supplier and is sold by EverXCloud Inc.