PyTorch 2.6.0 pre-built wheels for Linux are now compiled natively with CUDA 12.6 and cuDNN 9.2, replacing the previous 12.4-based wheels. 4. Performance Benchmarks (Reported from NVIDIA & PyTorch CI) Tests on 8x H100 GPUs with a Llama 2 70B model (TP=8):
| Operation | PyTorch 2.4 + CUDA 12.4 | PyTorch 2.6 + CUDA 12.6 | Improvement | |-----------|------------------------|-------------------------|-------------| | MFU (Model FLOPs utilization) | 38.2% | 40.5% | +2.3% | | Kernel launch time (microbench) | 12.4 µs | 8.2 µs | -34% | | cuDNN attention forward (512 seq len) | 0.43 ms | 0.39 ms | -9% |