Skip to content

PyTorch GPU Check Before Your First Run

Check CUDA, record your PyTorch environment and validate a training step before committing to a cloud GPU workload.

GuidePublished by EverXCloud Team4 sections2 min read

ShareXLinkedInReddit

Prepare the Complete Environment

Choose a GPU configuration for your model, not only a GPU name. Record GPU count and memory, host RAM, CPU, disk space, region and interconnect. Confirm that the installed PyTorch build supports the accelerator and driver. This walkthrough targets an NVIDIA CUDA environment with PyTorch already installed.

Use a disposable test environment you are authorized to run. The example below does not install packages, download a dataset or train a model. A successful CUDA check only establishes that PyTorch can access the device; it does not establish your training throughput or maximum model size.

Check CUDA and Record the Runtime

Run this after connecting to the test instance. Save the output alongside the container tag or image digest and the exact model and dataset revisions. If the check fails, review the selected environment and driver compatibility before starting a long job.

PyTorch CUDA environment check
nvidia-smi
python3 - <<'PY'
import json
import torch

if not torch.cuda.is_available():
    raise SystemExit("CUDA is unavailable: check the environment and driver.")

device = torch.cuda.current_device()
properties = torch.cuda.get_device_properties(device)
x = torch.ones((32, 32), device=device)
y = x @ x
torch.cuda.synchronize(device)
print(json.dumps({
    "torch_version": torch.__version__,
    "cuda_build": torch.version.cuda,
    "device": properties.name,
    "device_count": torch.cuda.device_count(),
    "total_memory_bytes": properties.total_memory,
    "matrix_check": y[0, 0].item(),
}, indent=2))
PY

Test One Representative Training Step

Load the actual model and a small representative batch, then run a forward pass, backward pass and optimizer step using the precision and sequence length you intend to use. A forward-only test misses gradients and optimizer states. Start with a small batch and record an out-of-memory failure rather than silently changing the workload between GPUs.

Warm up separately. Reset peak memory statistics immediately before the measured step, synchronize CUDA around timing and record both peak allocated and reserved memory. Include gradient accumulation, checkpointing, optimizer, device count and all software versions in the record. Test these choices again if you change them.

Measure Useful Work and Total Cost

Repeat the same workload and retain every measured run, including failures. Report examples or tokens processed per second only with the batch definition, sequence length and timing boundary. A small matrix check is not a training benchmark and must not be reported as one.

Track setup, data transfer and checkpoint time separately from steady training. Estimate total cost using the rate of the complete configuration, billable duration and retained resources. Save checkpoints and logs before terminating compute, then review any volumes that remain billable.

Sources

Keep Reading

All guides

Next guide2 min read

Benchmark vLLM Inference on a Cloud GPU

Prepare a local vLLM benchmark with fixed model, token lengths and concurrency. Record latency, failures and full session cost.

Request capacity

Get EverXCloud launch updates

New GPU capacity, regions, and marketplace features by email. Double opt-in, unsubscribe at any time.

Subscribe