Glossary3 sections1 min read
What is ROCm?
ROCm is AMD's open-source software platform for GPU computing, including drivers, compilers, the HIP programming interface and math libraries. It is what PyTorch and other frameworks use on AMD Instinct GPUs.
Why it matters when renting a GPU
AMD Instinct GPUs offer large memory per GPU, but your framework, kernels and inference server must support ROCm.
Test your exact model and software versions before committing to a long rental on a ROCm machine.
Worked example
The AMD MI300X carries 192 GB of HBM3 at 5,300 GB/s, enough to hold the 140 GB of FP16 weights of a 70B model on a single GPU.
Frequently asked questions
- Does PyTorch support ROCm?
- Yes, PyTorch publishes ROCm builds. Check that your other dependencies (custom kernels, inference servers) support it too.