GPU and AI glossary

Plain-English definitions of the GPU and AI terms you meet when renting compute: HBM, NVLink, FP8, MIG, KV cache, GPU-hour and more.

A

Ada Lovelace: Ada Lovelace is the NVIDIA architecture used by the L40S and L4 data center GPUs, RTX Ada workstation cards and the GeForce RTX 40 series. It combines FP8 Tensor Cores with ray-tracing cores.
Ampere: Ampere is the NVIDIA architecture behind the A100, A30 and A10 data center GPUs, RTX A-series workstation cards and the GeForce RTX 30 series. It introduced TF32 and BF16 Tensor Core support and MIG.

B

Bare metal vs VM vs container: A bare-metal rental gives you the whole physical server. A virtual machine (VM) runs your own operating system on virtualized hardware with GPUs passed through. A container runs your software in an isolated environment on a host managed by the provider.
Batch size: Batch size is the number of samples (in training) or concurrent requests (in inference) processed together in one pass on the GPU.
BF16 (bfloat16): BF16, or bfloat16, is a 16-bit floating-point format with an 8-bit exponent (the same range as FP32) and a 7-bit mantissa. It is the usual precision for training and serving modern language models.
Blackwell: Blackwell is the NVIDIA GPU architecture that follows Hopper. It powers data center GPUs such as the B200, B300 and GB200, the RTX PRO Blackwell workstation cards and the GeForce RTX 50 series.

C

CUDA: CUDA is NVIDIA's parallel computing platform: a programming model, compiler, drivers and libraries (cuBLAS, cuDNN, NCCL) that let software run on NVIDIA GPUs.

D

Data center vs consumer GPU: Data center GPUs (A100, H100, H200, B200, L40S) are built for servers, with large memory, ECC, high-speed interconnects and features such as MIG. Consumer GPUs (GeForce RTX) are built for desktops, with less memory but strong performance for their price.

E

Egress: Egress is data leaving a provider's network, for example when you download checkpoints or results from a rented GPU server. Some clouds bill it separately.

F

Fine-tuning: Fine-tuning continues training a pretrained model on your own data so it adapts to a task, style or domain. Full fine-tuning updates every weight; parameter-efficient methods such as LoRA update only small adapters.
FP16 (half precision): FP16, or half precision, is a 16-bit floating-point format with a 5-bit exponent and a 10-bit mantissa. It halves memory compared with FP32 and runs on Tensor Cores.
FP32 and TF32: FP32 is standard 32-bit (4-byte) floating point. TF32 is a Tensor Core mode introduced with the A100 that keeps FP32's 8-bit exponent but only a 10-bit mantissa, so FP32 code runs much faster with little accuracy loss.
FP4: FP4 is a 4-bit floating-point format supported by NVIDIA Blackwell Tensor Cores, mainly for inference. It halves memory again compared with FP8, with accuracy depending on scaling and calibration.
FP64 (double precision): FP64, or double precision, stores each number in 64 bits (8 bytes). Scientific and engineering simulations often need it for numerical accuracy; deep learning almost never does.
FP8: FP8 is an 8-bit floating-point format (in E4M3 and E5M2 variants) supported by Tensor Cores from the Hopper and Ada Lovelace generations onward. It halves memory and roughly doubles Tensor Core throughput compared with 16-bit formats.

G

GPU-hour: A GPU-hour is one GPU used for one hour. GPU rental prices are quoted per GPU-hour, so a job's cost is the number of GPUs multiplied by the hours they run and the hourly price.
Grace Hopper and Grace Blackwell: Grace Hopper (GH200) and Grace Blackwell (GB200) are NVIDIA superchips that pair a Grace Arm CPU with Hopper or Blackwell GPUs over NVLink-C2C, a coherent link much faster than PCIe.

H

HBM (High Bandwidth Memory): HBM (High Bandwidth Memory) is DRAM stacked in layers and mounted beside the GPU die on the same package, connected through a very wide bus. It delivers much more memory bandwidth per GPU than the GDDR memory used on graphics cards.
HBM3e: HBM3e is an extended generation of HBM3 stacked memory with higher speed per pin and larger stacks. It is the memory used on the NVIDIA H200 and on Blackwell data center GPUs such as the B200.
Hopper: Hopper is the NVIDIA data center GPU architecture behind the H100 and H200. It introduced FP8 Tensor Cores and the Transformer Engine, which manages precision automatically for transformer models.

I

Inference vs training: Training computes and updates a model's weights from data; inference runs a trained model to produce outputs. Training is compute- and memory-heavy over long runs; inference is latency- and bandwidth-sensitive.
InfiniBand: InfiniBand is a high-bandwidth, low-latency network fabric used to connect servers in high-performance computing and multi-node GPU training clusters. It supports RDMA, so network cards can move data directly between machines' memory, including GPU memory with GPUDirect RDMA.

K

KV cache: The KV cache stores the attention keys and values computed for previous tokens so a language model does not recompute them at each new token. Its size grows with context length and batch size.

L

LoRA and QLoRA: LoRA (Low-Rank Adaptation) fine-tunes a model by training small low-rank adapter matrices while the original weights stay frozen. QLoRA does the same on top of a base model quantized to 4 bits.

M

Memory bandwidth: Memory bandwidth is the rate, in gigabytes per second, at which a GPU can read from and write to its own memory. It caps how fast memory-bound workloads such as LLM token generation can run.
MIG (Multi-Instance GPU): MIG (Multi-Instance GPU) is an NVIDIA feature of the A100, H100 and newer data center GPUs that partitions one physical GPU into up to seven isolated instances, each with its own memory and compute.
Model and tensor parallelism: Model parallelism splits one model across several GPUs. Tensor parallelism splits the matrices of each layer across GPUs; pipeline parallelism assigns different layers to different GPUs.

N

NVLink: NVLink is NVIDIA's high-speed point-to-point interconnect between GPUs. It lets GPUs exchange data directly, far faster than over the PCIe bus.
NVSwitch: NVSwitch is a switch chip that connects many GPUs over NVLink so that every GPU can talk to every other one at full NVLink speed, instead of only to its direct neighbors.

O

On-demand vs reserved: On-demand GPUs are rented by the hour with no commitment and released when you are done. Reserved GPUs are committed for a period, usually at a lower hourly rate, whether or not you use them.
Order book and limit order: An order book lists the live offers (sell prices) and bids (buy prices) for a GPU configuration. A limit order, or bid, is your instruction to rent at a maximum price per GPU-hour; it is matched when an offer meets it.

P

PCIe vs SXM: SXM is NVIDIA's mezzanine form factor: the GPU mounts on a baseboard with NVLink and a higher power budget. PCIe is the standard add-in card form factor that fits ordinary servers.

Q

Quantization (INT8, INT4): Quantization stores a model's weights (and sometimes activations) in fewer bits, such as 8-bit or 4-bit integers with scale factors, instead of 16-bit floating point. It cuts memory and often speeds up inference.

R

ROCm: ROCm is AMD's open-source software platform for GPU computing, including drivers, compilers, the HIP programming interface and math libraries. It is what PyTorch and other frameworks use on AMD Instinct GPUs.

S

Spot instance: On public clouds, a spot instance is spare capacity sold at a discount that the provider can reclaim with short notice. On EverXCloud, the spot exchange is a live market where GPU hours trade at the current best offer or at your bid.

T

TDP (power): TDP (thermal design power), or board power, is the maximum power a GPU is designed to draw and dissipate under load, in watts.
Tensor Core: A Tensor Core is a specialized unit inside NVIDIA GPUs that performs small matrix multiply-accumulate operations in one step. Deep learning frameworks use them for most of the math in training and inference.
TFLOPS: TFLOPS means trillions of floating-point operations per second. For AI GPUs, the headline figure is Tensor Core throughput in a given precision, quoted dense or with structured sparsity.
Tokens per second: Tokens per second is the rate at which a language model generates output tokens. It is measured per user stream (latency) or summed over all concurrent streams (throughput).

V

VRAM (GPU memory): VRAM is the memory physically attached to a GPU. A model, its activations and its cache must fit in VRAM (or be split across several GPUs) for the GPU to run it efficiently.

Be ready on launch day

Create your account now: you can top up your balance and launch a GPU as soon as the market opens.

Get priority access Explore the live market