Installation Procedure

STEP1. Add Official NVIDIA Repository #

Register the official NVIDIA CUDA repository for Rocky Linux 9 with your system. Running the following command will add the CUDA (NVIDIA) repository.

sudo dnf config-manager --add-repo \
  http://developer.download.nvidia.com/compute/cuda/repos/rhel9/$(uname -i)/cuda-rhel9.repo

After execution, the NVIDIA repository definition file will be added to /etc/yum.repos.d/, and you will be able to install packages from this repository going forward.

#

STEP2. Install Required Packages #

Install development packages required for building the NVIDIA driver and using CUDA. First, enable the EPEL (Extra Packages for Enterprise Linux) repository and the CRB (CodeReady Builder) repository which includes development packages.

This makes necessary components such as dkms available.

sudo dnf config-manager --set-enabled crb
sudo dnf install -y \
  https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm \
  https://dl.fedoraproject.org/pub/epel/epel-next-release-latest-9.noarch.rpm

Next, install kernel headers and development packages corresponding to the currently running kernel, as well as development tools.

The following command will install kernel headers, kernel development packages (kernel-devel), compilers (gcc/g++), DKMS, and other necessary tools ($(uname -r) will be replaced with your current kernel version).

sudo dnf install -y kernel-headers-$(uname -r) kernel-devel-$(uname -r) \
  make automake gcc gcc-c++ dkms pciutils elfutils-libelf-devel \
  libglvnd-devel libglvnd-opengl libglvnd-glx bzip2 tar pkgconfig acpid

Note: The above command specifies kernel headers and development packages matching the currently running kernel version. These will be installed to match Rocky Linux 9.3’s default kernel 5.14.0-362.8.1.el9_3.x86_64, allowing modules to be built against this kernel later. Installing the required packages in advance will make the subsequent driver installation proceed smoothly.

STEP3. Install NVIDIA Driver #

Use the DNF module feature to install the latest NVIDIA driver. The NVIDIA repository you added contains multiple driver module streams, but here we will select the latest DKMS version driver. Install the NVIDIA driver using the following command.

sudo dnf module install -y nvidia-driver:latest-dkms

If a GPG key import confirmation appears during execution, type “y” to accept it.

This will build and install the NVIDIA kernel module via DKMS and integrate it into your system. After installation is complete, you can verify the driver module information using the modinfo nvidia command.

STEP4. Install CUDA 12.8 #

Install NVIDIA CUDA Toolkit 12.8. The CUDA repository contains multiple CUDA-related metapackages, but we will use the cuda-toolkit-12-8 package which includes a complete set of tools needed for development (this includes CUDA 12.8 compiler and libraries, but does not include the driver).

Execute the following command to install CUDA Toolkit 12.8.

sudo dnf install -y cuda-toolkit-12-8

The above will install the compiler (nvcc), CUDA libraries, headers, and other components under /usr/local/cuda-12.6/.

To add the CUDA bin directory to the PATH environment variable, for example configure with echo 'export PATH=/usr/local/cuda-12.6/bin:$PATH' >> ~/.bashrc (also set LD_LIBRARY_PATH if needed).

STEP5. Apply System Configuration and Verify Operation #

Restart the system to apply the driver, and configure it to boot with the specified kernel. First, set GRUB to boot with kernel version 5.14.0-362.8.1.el9_3.x86_64 by default from the installed kernels. Execute the following command to set the appropriate kernel as the default boot entry.

sudo grubby --set-default /boot/vmlinuz-5.14.0-362.8.1.el9_3.x86_64

After running the above, you can verify that the default has been set using grubby --default-index or grubby --info=ALL. Next, restart the system. After restart, first verify the kernel version is 5.14.0-362.8.1.el9_3.x86_64 using uname -r. Also, confirm that the Nouveau driver is disabled as a precaution (if needed, check that lsmod | grep nouveau returns no output. If disabling is necessary, append blacklist nouveau to /etc/modprobe.d/blacklist-nouveau.conf and rebuild the initramfs using the dracut -f command).

Finally, verify that the NVIDIA GPU is correctly recognized and the driver is functioning. Use the lspci command to confirm the GPU is detected (running the command below will display NVIDIA GPU entries).

lspci | grep -i NVIDIA

If “NVIDIA Corporation …” entries are displayed from the above, the GPU is recognized on the PCI bus. Next, execute the nvidia-smi command to verify the NVIDIA driver status.

nvidia-smi

If the installation has completed successfully, information such as GPU list, driver version, and available memory will be displayed.

For example, if the driver corresponding to RTX 6000 Ada is correctly loaded, the GPU name and available CUDA version should be displayed. This completes the installation and configuration of the NVIDIA RTX 6000 Ada driver and CUDA 12.6. If necessary, restart again and verify there are no issues at boot.

Supplementary: About lspci and nvidia-smi Verification #

• lspci is a command that displays a list of PCI devices. By executing it as shown above, you can verify that the system recognizes an NVIDIA GPU.
• nvidia-smi is NVIDIA’s system management utility that allows you to verify that the installed driver is functioning correctly (i.e., that it recognizes the GPU).
• If GPU information can be retrieved with this command after driver installation, the installation is successful.