GuidePublished by EverXCloud Team4 sections2 min read
Prepare the Complete Environment
Choose a GPU configuration for your model, not only a GPU name. Record GPU count and memory, host RAM, CPU, disk space, region and interconnect. Confirm that the installed PyTorch build supports the accelerator and driver. This walkthrough targets an NVIDIA CUDA environment with PyTorch already installed.
Use a disposable test environment you are authorized to run. The example below does not install packages, download a dataset or train a model. A successful CUDA check only establishes that PyTorch can access the device; it does not establish your training throughput or maximum model size.
Check CUDA and Record the Runtime
Run this after connecting to the test instance. Save the output alongside the container tag or image digest and the exact model and dataset revisions. If the check fails, review the selected environment and driver compatibility before starting a long job.
nvidia-smi
python3 - <<'PY'
import json
import torch
if not torch.cuda.is_available():
raise SystemExit("CUDA is unavailable: check the environment and driver.")
device = torch.cuda.current_device()
properties = torch.cuda.get_device_properties(device)
x = torch.ones((32, 32), device=device)
y = x @ x
torch.cuda.synchronize(device)
print(json.dumps({
"torch_version": torch.__version__,
"cuda_build": torch.version.cuda,
"device": properties.name,
"device_count": torch.cuda.device_count(),
"total_memory_bytes": properties.total_memory,
"matrix_check": y[0, 0].item(),
}, indent=2))
PYTest One Representative Training Step
Load the actual model and a small representative batch, then run a forward pass, backward pass and optimizer step using the precision and sequence length you intend to use. A forward-only test misses gradients and optimizer states. Start with a small batch and record an out-of-memory failure rather than silently changing the workload between GPUs.
Warm up separately. Reset peak memory statistics immediately before the measured step, synchronize CUDA around timing and record both peak allocated and reserved memory. Include gradient accumulation, checkpointing, optimizer, device count and all software versions in the record. Test these choices again if you change them.
Measure Useful Work and Total Cost
Repeat the same workload and retain every measured run, including failures. Report examples or tokens processed per second only with the batch definition, sequence length and timing boundary. A small matrix check is not a training benchmark and must not be reported as one.
Track setup, data transfer and checkpoint time separately from steady training. Estimate total cost using the rate of the complete configuration, billable duration and retained resources. Save checkpoints and logs before terminating compute, then review any volumes that remain billable.
Sources
- PyTorch CUDA semantics (checked October 28, 2026)
- PyTorch quickstart (checked October 28, 2026)
