From 0f7fb7f78a530f052e9874345abfc05819383f84 Mon Sep 17 00:00:00 2001 From: Emma Thorpe Date: Mon, 17 Aug 2026 20:35:39 +0100 Subject: [PATCH] feat(macpro31): NVIDIA Quadro P400 driver and CUDA-enabled Docker The stock GPU has been replaced with a Quadro P400 (Pascal, GP108). Add hosts/MacPro31/nvidia.nix: - Driver branch 580 (nvidiaPackages.legacy_580), not the nixpkgs default production branch (595.x). 580 is the last branch supporting Maxwell/Pascal/Volta and is an LTS branch until Aug 2028; a newer one does not drive this card. - modesetting.enable for Wayland (nvidia-drm.modeset=1), open = false (the open kernel modules need Turing or later), and sway --unsupported-gpu, which wlroots requires with the proprietary driver. - Docker with GPU access via CDI (hardware.nvidia-container-toolkit), rather than the deprecated virtualisation.docker.enableNvidia runtime wrapper. Containers run with --device=nvidia.com/gpu=all and must ship a CUDA 12.x or older runtime: CUDA 13 dropped sm_61. The driver packages are unfree, so allowlist them in unfreePackages; they are not cached and the kernel module builds on the host. Also declare features.cpu.microarchLevel = 1 for this machine: the Harpertown Xeons have SSE4.1 but no SSE4.2/POPCNT, which switches off Claude Code through the fleet-wide gate. --- flake.nix | 6 +++ hosts/MacPro31/README.md | 73 +++++++++++++++++++++++++++----- hosts/MacPro31/configuration.nix | 20 ++++----- 3 files changed, 77 insertions(+), 22 deletions(-) diff --git a/flake.nix b/flake.nix index d982c83..d8109f4 100644 --- a/flake.nix +++ b/flake.nix @@ -111,8 +111,14 @@ ]; # Unfree packages permitted to be built (replaces blanket allowUnfree). + # The NVIDIA entries are for the Mac Pro's Quadro P400 (hosts/MacPro31/ + # nvidia.nix); unfree packages are not in the binary cache, so the + # kernel module is compiled on the host. unfreePackages = [ "claude-code" + "nvidia-x11" + "nvidia-kernel-modules" + "nvidia-settings" ]; # Per-user identity, keyed by username. See README "Users". diff --git a/hosts/MacPro31/README.md b/hosts/MacPro31/README.md index 8b5ce00..dfc8a6c 100644 --- a/hosts/MacPro31/README.md +++ b/hosts/MacPro31/README.md @@ -1,7 +1,7 @@ # Mac Pro 3,1 (Early 2008) — install notes Flake host: `lyrathorpe-macpro31`. Desktop (`portable = false`, imports -`../../modules/desktop.nix`). Files: `configuration.nix`, +`../../modules/desktop.nix`). Files: `configuration.nix`, `nvidia.nix`, `hardware-configuration.nix`. ## Hardware configuration @@ -27,18 +27,69 @@ either Partition the disk GPT with an ESP (vfat). -## Graphics +## Graphics — NVIDIA Quadro P400 -The stock card varies between units — **ATI Radeon HD 2600 XT** or **NVIDIA -GeForce 8800 GT**. No proprietary driver is hardcoded; Sway relies on in-tree KMS: +The stock card (**ATI Radeon HD 2600 XT** or **NVIDIA GeForce 8800 GT**, +depending on the unit) has been replaced with an **NVIDIA Quadro P400** (Pascal, +GP108). Everything driver-related lives in [`nvidia.nix`](./nvidia.nix): -- ATI Radeon HD 2600 XT → `radeon` (or `amdgpu`) KMS -- NVIDIA GeForce 8800 GT → `nouveau` KMS +- **Driver branch 580** (`nvidiaPackages.legacy_580`), _not_ the nixpkgs default + (`production`, currently 595.x). 580 is the last branch that supports + Maxwell/Pascal/Volta and is maintained as an LTS branch until Aug 2028; a + newer branch does not drive this card at all. +- `modesetting.enable = true` — mandatory for Wayland (sets + `nvidia-drm.modeset=1`); without it wlroots gets no GBM device and both Sway + and the greeter fail to start. +- `open = false` — the open kernel modules require Turing or later. +- Sway runs with `--unsupported-gpu` (`programs.sway.extraOptions`); wlroots + refuses the proprietary driver otherwise. `cage`/ReGreet needs no such flag. +- nouveau and `nvidiafb` are blacklisted automatically by the NVIDIA module. -These come up automatically. If a card needs forcing, set -`services.xserver.videoDrivers` and/or add the module to -`boot.initrd.kernelModules` for early KMS (see the comment in -`configuration.nix`). +The driver is unfree, so it is **not in the binary cache**: the kernel module is +compiled on the machine, which on these 2008 Xeons is slow — budget for a long +first rebuild and again after every kernel bump. The package names are +allowlisted in `unfreePackages` in [`flake.nix`](../../flake.nix). + +Note the Mac Pro shows no EFI boot screen with a stock PC card (no Apple EFI +ROM): the machine boots blind until KMS brings the display up. That is expected, +not a fault. + +Verify after a rebuild: + +```sh +nvidia-smi +``` + +## Docker with CUDA + +`nvidia.nix` also enables Docker and gives containers GPU access via **CDI** +(`hardware.nvidia-container-toolkit.enable`), which generates device specs from +the host driver at boot (regenerated by a udev rule when the `nvidia` device +appears) and turns on the daemon's CDI feature: + +```sh +docker run --rm --device=nvidia.com/gpu=all nvidia/cuda:12.9.1-base-ubuntu24.04 nvidia-smi +``` + +- Use the `--device=nvidia.com/gpu=all` form. `--gpus all` is the legacy + runtime-wrapper path (`virtualisation.docker.enableNvidia`), which is + deprecated upstream and deliberately not enabled here. +- **CUDA version matters.** The P400 is compute capability 6.1 (`sm_61`); CUDA + 13 dropped Maxwell/Pascal/Volta, so container images must ship a **CUDA 12.x + or older** runtime. The 580 driver itself is happy with either. +- 2 GB of VRAM, 256 CUDA cores — fine for encode/decode and small models, not + for training anything serious. +- Docker socket is local-only (no TCP listener, unlike the Pi). Users need the + `docker` group; the registry already grants it. + +## Claude Code — not installed here + +The dual Harpertown Xeons are **x86-64-v1** (SSE4.1, but no SSE4.2/POPCNT) and +the Node runtime Claude Code ships on requires x86-64-v2. `configuration.nix` +declares `features.cpu.microarchLevel = 1`, which switches the tool off through +the fleet-wide gate in [`../../modules/features.nix`](../../modules/features.nix) +— see the root README. Forcing `features.claudeCode.enable` on here is an +evaluation error, not a broken install. ## Networking @@ -52,7 +103,7 @@ Graphical login via a Wayland greeter — `greetd` running ReGreet inside the every Sway host (gated on `features.swayDesktop.enable`). The greeter is forced to the Dvorak layout to match the console and Sway session. Set the user password (`passwd lyrathorpe`) after install, or the greeter cannot -authenticate. Requires working KMS (radeon/nouveau — see Graphics). +authenticate. Requires working KMS (NVIDIA modesetting — see Graphics). ## Apply diff --git a/hosts/MacPro31/configuration.nix b/hosts/MacPro31/configuration.nix index 6faae6d..82eee35 100644 --- a/hosts/MacPro31/configuration.nix +++ b/hosts/MacPro31/configuration.nix @@ -7,8 +7,14 @@ { imports = [ ./hardware-configuration.nix + ./nvidia.nix ]; + # Dual quad-core Xeon (Harpertown/Penryn): SSE4.1 but no SSE4.2 or POPCNT, + # i.e. x86-64-v1. Declaring it here switches off the fleet flags that need a + # newer CPU -- currently features.claudeCode (see ../../modules/features.nix). + features.cpu.microarchLevel = 1; + # The Mac Pro 3,1 has 64-bit EFI (confirmed by the owner), so boot via # systemd-boot like the MBP -- no GRUB/BIOS shim needed. boot.loader.systemd-boot.enable = true; @@ -33,17 +39,9 @@ # enabled in workstation.nix. hardware.cpu.intel.updateMicrocode = true; - # GPU note: the stock card varies between units -- ATI Radeon HD 2600 XT or - # NVIDIA GeForce 8800 GT. Sway needs a working KMS/modesetting driver; do NOT - # install a proprietary blob here. Depending on the installed card, rely on - # the open kernel driver: - # - ATI Radeon HD 2600 XT -> "radeon" (older) or "amdgpu" KMS - # - NVIDIA GeForce 8800 GT -> "nouveau" KMS - # These come up automatically via the in-tree drivers + KMS, and the graphics - # stack itself is enabled by modules/sway.nix. If a card needs to be forced, add it - # here, e.g. `services.xserver.videoDrivers = [ "radeon" ];` (or "nouveau"), - # and/or `boot.initrd.kernelModules = [ "radeon" ];` in - # hardware-configuration.nix for early KMS. + # GPU: the stock card (ATI Radeon HD 2600 XT / NVIDIA GeForce 8800 GT) has + # been replaced with an NVIDIA Quadro P400. Driver, Wayland quirks and + # GPU-enabled Docker live in ./nvidia.nix. # See `man configuration.nix` / the stateVersion docs before changing. system.stateVersion = "26.05";