fix(macpro31): load the NVIDIA modules and guard the CDI generator
CI / flake (push) Skipped
CI / flake (pull_request) Successful in 4m12s
CI / flake (push) Skipped
CI / flake (pull_request) Successful in 4m12s
The CDI generator aborted with "failed to initialize NVML: Driver Not Loaded", taking docker.service with it (requiredBy) and failing the switch. Two causes. The nixpkgs NVIDIA module only adds nvidia/nvidia_modeset/ nvidia_drm to boot.kernelModules when services.xserver.enable is set, which is false on this Wayland-only host, so load them explicitly. nvidia_uvm stays out: the module's modprobe softdep loads it once the GPU device exists. The generator also runs during activation, when a module rebuilt against a new kernel cannot be loaded until reboot -- a guaranteed failure after every kernel bump. Guard it with ConditionPathExists on /proc/driver/nvidia/version so it skips rather than fails; the toolkit's udev rule restarts it when the device appears, so the specs are generated on the next boot.
This commit is contained in:
@@ -82,6 +82,32 @@ docker run --rm --device=nvidia.com/gpu=all nvidia/cuda:12.9.1-base-ubuntu24.04
|
||||
- Docker socket is local-only (no TCP listener, unlike the Pi). Users need the
|
||||
`docker` group; the registry already grants it.
|
||||
|
||||
### "Driver Not Loaded" from the CDI generator
|
||||
|
||||
`nvidia-container-toolkit-cdi-generator.service` fails with
|
||||
`failed to initialize NVML: Driver Not Loaded` whenever the `nvidia` kernel
|
||||
module is not loaded in the **running** kernel. After a kernel bump that is
|
||||
unavoidable — the rebuilt module cannot load until reboot — so the unit is
|
||||
guarded with `ConditionPathExists=/proc/driver/nvidia/version` and skips
|
||||
instead of failing. Without that guard it also takes `docker.service`
|
||||
(`requiredBy`) with it and makes `nixos-rebuild switch` exit non-zero.
|
||||
|
||||
**Reboot after a rebuild that touches the driver or the kernel.** The toolkit's
|
||||
udev rule restarts the generator when the GPU device appears, so the CDI specs
|
||||
are written on the next boot. To check the state:
|
||||
|
||||
```sh
|
||||
lsmod | grep nvidia # nvidia, nvidia_modeset, nvidia_drm, nvidia_uvm
|
||||
cat /proc/driver/nvidia/version
|
||||
nvidia-smi
|
||||
systemctl status nvidia-container-toolkit-cdi-generator.service
|
||||
ls /var/run/cdi # the generated spec
|
||||
```
|
||||
|
||||
If the module is genuinely absent after a reboot, check `dmesg | grep -i
|
||||
nvidia` (build/version mismatch, or nouveau still bound — the module blacklists
|
||||
it, so that should not happen).
|
||||
|
||||
## Claude Code — not installed here
|
||||
|
||||
The dual Harpertown Xeons are **x86-64-v1** (SSE4.1, but no SSE4.2/POPCNT) and
|
||||
|
||||
@@ -28,6 +28,18 @@
|
||||
open = false;
|
||||
};
|
||||
|
||||
# The NVIDIA module only puts these in boot.kernelModules when
|
||||
# services.xserver.enable is true, which is false on this Wayland-only host --
|
||||
# so load them explicitly rather than relying on udev modalias autoloading.
|
||||
# nvidia_uvm (needed by CUDA) is deliberately absent: the module's modprobe
|
||||
# softdep pulls it in after the GPU device exists, which is the supported
|
||||
# ordering.
|
||||
boot.kernelModules = [
|
||||
"nvidia"
|
||||
"nvidia_modeset"
|
||||
"nvidia_drm"
|
||||
];
|
||||
|
||||
# wlroots refuses the proprietary NVIDIA driver unless told to proceed. The
|
||||
# greeter's compositor (cage) has no such check; only Sway needs the flag,
|
||||
# which the module bakes into the wrapper the session's .desktop file runs.
|
||||
@@ -40,4 +52,15 @@
|
||||
# with `docker run --device=nvidia.com/gpu=all ...`. The deprecated
|
||||
# virtualisation.docker.enableNvidia runtime wrapper is deliberately not used.
|
||||
hardware.nvidia-container-toolkit.enable = true;
|
||||
|
||||
# The generator needs a loaded kernel module: without one it aborts with
|
||||
# "failed to initialize NVML: Driver Not Loaded". That is guaranteed after a
|
||||
# kernel bump, where the rebuilt module cannot load until reboot -- and since
|
||||
# the unit is requiredBy docker.service and wantedBy multi-user.target, the
|
||||
# failure takes Docker down and makes `nixos-rebuild switch` exit non-zero.
|
||||
# Skip the run instead when no driver is loaded; the toolkit's udev rule
|
||||
# restarts the unit as soon as the nvidia device appears, so the CDI specs are
|
||||
# still generated on the next boot.
|
||||
systemd.services.nvidia-container-toolkit-cdi-generator.unitConfig.ConditionPathExists =
|
||||
"/proc/driver/nvidia/version";
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user