Glossary3 sections1 min read
What is FP8?
FP8 is an 8-bit floating-point format (in E4M3 and E5M2 variants) supported by Tensor Cores from the Hopper and Ada Lovelace generations onward. It halves memory and roughly doubles Tensor Core throughput compared with 16-bit formats.
Why it matters when renting a GPU
FP8 inference lets a model use half the memory of BF16 with careful calibration, and FP8 training is supported by recent frameworks.
Older GPUs such as the A100 have no FP8 Tensor Core support; check the FP8 figure on the GPU page.
Worked example
The H100 SXM is rated at 1,979 dense FP8 TFLOPS against 989 in BF16, and the L40S at 733 against 362. A 70B model in FP8 needs about 70 × 1 = 70 GB of weights.
Frequently asked questions
- Does the A100 support FP8?
- No. FP8 Tensor Core support starts with Hopper (H100) and Ada Lovelace (L40S, L4, RTX 4090).