Glossary3 sections1 min read
What is FP16 (half precision)?
FP16, or half precision, is a 16-bit floating-point format with a 5-bit exponent and a 10-bit mantissa. It halves memory compared with FP32 and runs on Tensor Cores.
Why it matters when renting a GPU
Model sizes are often quoted in FP16: 2 bytes per parameter is the baseline for estimating VRAM.
FP16 training needs loss scaling to avoid underflow; inference in FP16 is straightforward.
Worked example
A 70B-parameter model in FP16 needs 70 × 2 = 140 GB for its weights, which does not fit on one 80 GB H100 but fits on one 192 GB MI300X.
Frequently asked questions
- Is FP16 enough for inference?
- Yes for most models. Many deployments go further with 8-bit or 4-bit quantization to save memory.