Glossary3 sections1 min read
What is FP4?
FP4 is a 4-bit floating-point format supported by NVIDIA Blackwell Tensor Cores, mainly for inference. It halves memory again compared with FP8, with accuracy depending on scaling and calibration.
Why it matters when renting a GPU
FP4 lets very large models run on fewer GPUs, at the cost of careful quantization and validation of output quality.
Only Blackwell-generation GPUs (B200, B300, GB200, RTX 50 series, RTX PRO Blackwell) run FP4 natively on Tensor Cores.
Worked example
At half a byte per parameter, a 405B-parameter model needs about 405 × 0.5 = 202.5 GB of weights in FP4, against 810 GB in BF16.
Frequently asked questions
- Is FP4 the same as INT4?
- No. Both use 4 bits, but FP4 is a floating-point format with an exponent, while INT4 is an integer format with a scale factor.