nixfiles/hosts/MacPro31/README.md

# Mac Pro 3,1 (Early 2008) — install notes

Flake host: `lyrathorpe-macpro31`. Desktop (`portable = false`, imports
`../../modules/desktop.nix`). Files: `configuration.nix`, `nvidia.nix`,
`hardware-configuration.nix`.

## Hardware configuration

`hardware-configuration.nix` here is the real config generated by
`nixos-generate-config` on the machine. Root is an **LVM** logical volume
(`/dev/mapper/MacPro-Root`, ext4); the ESP (vfat) and swap are referenced by
UUID. The initrd carries `dm-snapshot` for the LVM root. Regenerate and commit
if the disk layout changes.

## Bootloader

The Mac Pro 3,1 has **64-bit EFI**, so it uses **systemd-boot** (no GRUB/CSM
shim). `canTouchEfiVariables = false` because Apple's firmware does not reliably
accept `efibootmgr` NVRAM writes.

Apple-EFI quirk: if the firmware boot picker does not show NixOS after install,
either

- uncomment `boot.loader.efi.efiInstallAsRemovable = true;` in
  `configuration.nix` (installs the fallback `\EFI\BOOT\BOOTX64.EFI`), and/or
- "bless" the ESP from macOS.

Partition the disk GPT with an ESP (vfat).

## Graphics — NVIDIA Quadro P400

The stock card (**ATI Radeon HD 2600 XT** or **NVIDIA GeForce 8800 GT**,
depending on the unit) has been replaced with an **NVIDIA Quadro P400** (Pascal,
GP108). Everything driver-related lives in [`nvidia.nix`](./nvidia.nix):

- **Driver branch 580** (`nvidiaPackages.legacy_580`), _not_ the nixpkgs default
  (`production`, currently 595.x). 580 is the last branch that supports
  Maxwell/Pascal/Volta and is maintained as an LTS branch until Aug 2028; a
  newer branch does not drive this card at all.
- `modesetting.enable = true` — mandatory for Wayland (sets
  `nvidia-drm.modeset=1`); without it wlroots gets no GBM device and both Sway
  and the greeter fail to start.
- `open = false` — the open kernel modules require Turing or later.
- Sway runs with `--unsupported-gpu` (`programs.sway.extraOptions`); wlroots
  refuses the proprietary driver otherwise. `cage`/ReGreet needs no such flag.
- nouveau and `nvidiafb` are blacklisted automatically by the NVIDIA module.

The driver is unfree, so it is **not in the binary cache**: the kernel module is
compiled on the machine, which on these 2008 Xeons is slow — budget for a long
first rebuild and again after every kernel bump. The package names are
allowlisted in `unfreePackages` in [`flake.nix`](../../flake.nix).

Note the Mac Pro shows no EFI boot screen with a stock PC card (no Apple EFI
ROM): the machine boots blind until KMS brings the display up. That is expected,
not a fault.

Verify after a rebuild:

```sh
nvidia-smi
```

## Docker with CUDA

`nvidia.nix` also enables Docker and gives containers GPU access via **CDI**
(`hardware.nvidia-container-toolkit.enable`), which generates device specs from
the host driver at boot (regenerated by a udev rule when the `nvidia` device
appears) and turns on the daemon's CDI feature:

```sh
docker run --rm --device=nvidia.com/gpu=all nvidia/cuda:12.9.1-base-ubuntu24.04 nvidia-smi
```

- Use the `--device=nvidia.com/gpu=all` form. `--gpus all` is the legacy
  runtime-wrapper path (`virtualisation.docker.enableNvidia`), which is
  deprecated upstream and deliberately not enabled here.
- **CUDA version matters.** The P400 is compute capability 6.1 (`sm_61`); CUDA
  13 dropped Maxwell/Pascal/Volta, so container images must ship a **CUDA 12.x
  or older** runtime. The 580 driver itself is happy with either.
- 2 GB of VRAM, 256 CUDA cores — fine for encode/decode and small models, not
  for training anything serious.
- Docker socket is local-only (no TCP listener, unlike the Pi). Users need the
  `docker` group; the registry already grants it.

### "Driver Not Loaded" from the CDI generator

`nvidia-container-toolkit-cdi-generator.service` fails with
`failed to initialize NVML: Driver Not Loaded` whenever the `nvidia` kernel
module is not loaded in the **running** kernel. After a kernel bump that is
unavoidable — the rebuilt module cannot load until reboot — so the unit is
guarded with `ConditionPathExists=/proc/driver/nvidia/version` and skips
instead of failing. Without that guard it also takes `docker.service`
(`requiredBy`) with it and makes `nixos-rebuild switch` exit non-zero.

**Reboot after a rebuild that touches the driver or the kernel.** The toolkit's
udev rule restarts the generator when the GPU device appears, so the CDI specs
are written on the next boot. To check the state:

```sh
lsmod | grep nvidia            # nvidia, nvidia_modeset, nvidia_drm, nvidia_uvm
cat /proc/driver/nvidia/version
nvidia-smi
systemctl status nvidia-container-toolkit-cdi-generator.service
ls /var/run/cdi                # the generated spec
```

If the module is genuinely absent after a reboot, check `dmesg | grep -i
nvidia` (build/version mismatch, or nouveau still bound — the module blacklists
it, so that should not happen).

## Claude Code — not installed here

The dual Harpertown Xeons are **x86-64-v1** (SSE4.1, but no SSE4.2/POPCNT) and
the Node runtime Claude Code ships on requires x86-64-v2. `configuration.nix`
declares `features.cpu.microarchLevel = 1`, which switches the tool off through
the fleet-wide gate in [`../../modules/features.nix`](../../modules/features.nix)
— see the root README. Forcing `features.claudeCode.enable` on here is an
evaluation error, not a broken install.

## Networking

Wired Ethernet via NetworkManager (from `desktop.nix`) — the Mac Pro has two
gigabit ports.

## Login

Graphical login via a Wayland greeter — `greetd` running ReGreet inside the
`cage` kiosk compositor — configured centrally in `../../modules/sway.nix` for
every Sway host (gated on `features.swayDesktop.enable`). The greeter is forced
to the Dvorak layout to match the console and Sway session. Set the user
password (`passwd lyrathorpe`) after install, or the greeter cannot
authenticate. Requires working KMS (NVIDIA modesetting — see Graphics).

## Apply

```sh
sudo nixos-rebuild switch --flake .#lyrathorpe-macpro31
```