Glossary3 sections1 min read
What is Tensor Core?
A Tensor Core is a specialized unit inside NVIDIA GPUs that performs small matrix multiply-accumulate operations in one step. Deep learning frameworks use them for most of the math in training and inference.
Why it matters when renting a GPU
The TFLOPS figures quoted for AI GPUs are Tensor Core figures in a given precision. Each generation adds faster formats such as FP8 or FP4.
Software must use a supported precision to benefit: running in FP32 leaves most Tensor Core throughput unused.
Worked example
The H100 SXM is rated at 989 dense FP16 Tensor TFLOPS, against 65 on the Tesla T4: about 15.2 times more.
Frequently asked questions
- Do AMD GPUs have Tensor Cores?
- Tensor Core is an NVIDIA name. AMD Instinct GPUs have their own matrix units (Matrix Cores) for the same purpose.