Merge pull request 'feat: cold report, playlist pruning, and .m3u8 playlists' (#13) from feat/cold-report into main

Reviewed-on: #13

.gitea/workflows/build-and-publish.yaml

@@ -45,7 +45,8 @@ jobs:
 
       # The suite runs inside the image, against the interpreter that ships,
       # rather than against whatever the runner happens to provide. A failing
-      # test fails the build. Layers are shared with the push build below.
+      # test fails the build. The runtime stage below is built from the same
+      # daemon afterwards, so its layers are already in cache.
       - name: Run the test suite inside the image
         run: docker build --target test -t music-curator:test .
 
@@ -124,9 +125,6 @@ jobs:
           echo "release=${release}" >> "$GITHUB_OUTPUT"
           echo "Computed bump=${bump}, release=${release}, base=${base}"
 
-      - name: Set up Buildx
-        uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4
-
       - name: Log in to the Gitea container registry
         if: github.event_name != 'pull_request'
         uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4
@@ -135,21 +133,33 @@ jobs:
           username: ${{ github.repository_owner }}
           password: ${{ secrets.PACKAGES_TOKEN }}
 
-      - name: Build and push
-        uses: docker/build-push-action@f9f3042f7e2789586610d6e8b85c8f03e5195baf # v7
-        with:
-          context: .
-          # Without this the last stage in the Dockerfile -- the test stage --
-          # would be what gets published.
-          target: runtime
-          # The NAS is the only host this runs on. Building arm64 as well would
-          # mean emulating it under QEMU for no consumer.
-          platforms: linux/amd64
-          push: ${{ github.event_name != 'pull_request' }}
-          tags: ${{ steps.version.outputs.tags }}
-          labels: |
-            org.opencontainers.image.source=${{ github.server_url }}/${{ github.repository }}
-            org.opencontainers.image.revision=${{ github.sha }}
+      # Plain `docker build` rather than buildx. buildx boots its own buildkit
+      # in a container with a cache of its own, so it shared nothing with the
+      # test build above and rebuilt the image from the base image up -- two
+      # full builds per run. It earns that cost when building for several
+      # platforms; this only ever targets the amd64 NAS, so it does not.
+      #
+      # `--target runtime` is a strict prefix of the test stage, so every layer
+      # is already in the daemon's cache and this resolves in seconds.
+      - name: Build the runtime image
+        run: |
+          set -euo pipefail
+          tags=()
+          while IFS= read -r tag; do
+            [ -n "$tag" ] && tags+=(-t "$tag")
+          done <<< "${{ steps.version.outputs.tags }}"
+          docker build --target runtime \
+            --label "org.opencontainers.image.source=${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}" \
+            --label "org.opencontainers.image.revision=${GITHUB_SHA}" \
+            "${tags[@]}" .
+
+      - name: Push
+        if: github.event_name != 'pull_request'
+        run: |
+          set -euo pipefail
+          while IFS= read -r tag; do
+            [ -n "$tag" ] && docker push "$tag"
+          done <<< "${{ steps.version.outputs.tags }}"
 
       # Record the release: write the computed version into pyproject.toml, then
       # commit and tag it, so the packaging metadata always matches the release

README.md

@@ -7,22 +7,315 @@ which keeps an MP3 copy of a lossless library for an iPod. This one answers the
 question that mirror cannot: which of it is worth carrying, and which of it has
 not been played in years.
 
-**This is stage one.** It ingests the scrobble history and nothing else. There
-are no playlists yet, and nothing touches Lidarr or the music library. See
-"Where this is going" below.
+**This is stage four, read-only.** It ingests the scrobble history, indexes the library
+from Lidarr, matches one to the other, and writes playlists into the mirror —
+by listening history and by mood.
+Nothing is written back to Lidarr — every call there is a `GET`. See "Where this
+is going" below.
 
 ## What it does today
 
 - Pulls the full Last.fm scrobble history into SQLite, then keeps it current.
 - Tracks loved tracks separately, as the protected set for later stages.
-- Reports what it holds, including the number that matters most: how many
-  scrobbles carry a MusicBrainz recording id.
+- Indexes every artist, album and track Lidarr knows about, with file paths and
+  the date each file landed.
+- Ties the two together and reports how well it managed.
+- Writes M3U playlists into the mirror, from the listening history and by mood.
+- Reports which albums have never been played. It does not act on that.
 
-That last figure decides the next stage. Lidarr exposes a `ForeignRecordingId`
-on every track, which is the same identifier, so scrobbles carrying one can be
-joined to the library exactly. The rest have to go through name matching, which
-is where a curation tool goes wrong and starts recommending the deletion of
-music you love. Measure the join rate before trusting the verdict.
+## Matching
+
+Two tiers, and no third.
+
+| Tier | Key | Notes |
+| ------ | --------------------------- | ----------------------------------------- |
+| `mbid` | MusicBrainz recording id | Exact. Last.fm's per-scrobble `mbid` against Lidarr's `ForeignRecordingId` |
+| `name` | Normalised artist and title | Everything the first tier could not carry |
+| `none` | — | Recorded as a miss, never guessed at |
+
+The name tier does almost all of the work. A recording MBID is exact when it
+lands, but MusicBrainz holds a separate recording per release, and Last.fm and
+Lidarr rarely pick the same one: on a real library, two thirds of scrobbles
+carry a recording id and barely a twentieth of them join on it. The report
+counts how many carried an id and matched on name anyway, which is the measure
+of that disagreement.
+
+So the normalisation is the load-bearing part, because the two sides disagree in
+predictable ways. It folds case and accents, drops guest credits (`Yellowcard
+feat. Tay Jardine` against a tag of `Yellowcard`), strips a trailing
+version suffix (`(Remastered 2011)`, `- Live`), expands `&`, and removes a
+leading `The`. Punctuation gets two different rules that pull against each
+other and are both required: apostrophes are **deleted**, so `Don't` meets
+`Dont`, while every other mark becomes a **space**, so `AC/DC`, `AC-DC` and
+`AC DC` all meet as well.
+
+It leans towards collapsing too much. A false match makes something look
+played; a missed match makes something look abandoned. Only one of those
+deletes music.
+
+### Indexing quirks
+
+Albums are fetched from the **unfiltered** `GET /api/v1/album` first: one
+request, and the only path that skips albums whose artist metadata is missing
+rather than dereferencing it.
+
+That is not enough on its own. Every album endpoint maps through a resource
+that picks the release with `SingleOrDefault(x => x.Monitored)`, which throws
+for an album with **two monitored releases** and takes the whole response with
+it:
+
+```
+HTTP 500: Sequence contains more than one element
+```
+
+When the bulk call dies that way, the indexer falls back to one request per
+artist. It cannot avoid the exception, but it confines it to whichever artist
+owns the offending album and names them in the log — which is the only
+practical way to find it in a large library. Open that artist in Lidarr and
+check the Releases tab of each album: exactly one release may be monitored.
+
+Losing an artist's albums does not cost their tracks, which come from a
+different endpoint with a different mapper, so matching is unaffected. A cull
+would not be, and the report says so.
+
+### Talking to Lidarr
+
+Indexing is two requests per artist, and more when the album fallback fires. On
+a large library that is thousands of requests in a few minutes. `urllib` opens a
+new TCP connection and performs a new DNS lookup for every one of them, which is
+enough to exhaust a container's resolver and produce `[Errno -3] Try again` on
+everything at once. The client therefore holds one connection open per host and
+resolves once.
+
+Transient failures — a dropped connection, a resolver hiccup, `429`, `502`,
+`503`, `504` — are retried with a backoff. An HTTP `500` is not: it is an
+unhandled exception inside Lidarr's own serialisation and will be raised again
+identically. That distinction also decides whether a failure is worth
+investigating; a library-wide outage is not probed artist by artist, because
+doing so multiplies the load that caused it.
+
+A local address is preferable to a public hostname here. It removes DNS, the
+reverse proxy and its timeouts from a path that needs none of them.
+
+Tracks and files have no unfiltered endpoint — Lidarr rejects a call with no
+filter — so they stay per artist. If one artist cannot be served, that artist is
+skipped and the run continues, but the count is recorded and the coverage report
+says so loudly. A missing artist makes their played music look cold, so an
+incomplete index must never be culled against.
+
+### Reading the coverage report
+
+Matched against unmatched is the wrong comparison — most unmatched listening is
+music that was never in the library, which says nothing at all about the
+matcher. The line to watch is:
+
+```
+unmatched by an artist the library holds: N pairs, M plays
+```
+
+That is a track that was played by an artist the library holds. It is then
+split three ways, because owning an artist is a weak proxy for owning a track
+and a shared title is a weak proxy for a shared song:
+
+- **the library's own title credits the scrobbled artist** — `Voodoo People
+  (Pendulum Remix)` against a play credited to Pendulum. Same song, filed
+  under the original artist. These are the genuine misses.
+- **the same title under an unrelated artist** — a collision, not a miss.
+  Across fifty thousand tracks these are constant: `Everyday` is Rusko and
+  also Def Leppard, `Kaleidoscope` is Delta Heavy and also Chappell Roan.
+  Matching on title alone would be far worse than missing them, which is why
+  there is no such tier.
+- **the title is nowhere in the library** — never bought.
+
+Only the first is worth chasing. Counting all three as matcher failures
+overstates the problem and would over-block the cull.
+
+## Playlists
+
+Written into `<mirror>/_playlists/` as extended M3U, rebuilt every pass.
+
+The extension is **`.m3u8`**, not `.m3u`. Rockbox's `is_m3u8_name()` treats
+every extension as UTF-8 *except* an explicit `.m3u`, which it decodes through
+the user's configured codepage instead — so a plain `.m3u` mangles every
+accented filename. No byte order mark is written: Rockbox does not need one at
+this extension, and a BOM upsets players that do not expect it. Six
+rules, capped at `--playlist-limit` tracks each:
+
+| Playlist | Rule |
+| -------------------- | --------------------------------------------------------- |
+| `heavy-rotation` | Most played over the last twelve months |
+| `all-time` | Most played ever |
+| `neglected` | Played heavily once, silent for twelve months |
+| `deep-cuts` | Never played, from albums whose other tracks you play constantly |
+| `unheard-favourites` | Never played, by the artists you play most |
+| `unheard` | Never played, anywhere in the library |
+
+Ninety days was the obvious window for "recent" and is the wrong one: on a real
+history it holds a few hundred plays spread thinly across a twenty-thousand
+track rotation, so nothing ranks meaningfully. Twelve months does.
+
+The two `unheard` playlists rotate **weekly**, not per pass. A pass runs every
+few hours, and a playlist that reorders itself each time is one that has to be
+re-imported each time — the Music app imports a snapshot of a file, it does not
+track it.
+
+### Moods
+
+A second set of playlists selects by **Last.fm's crowd tags** rather than by
+listening history. MusicBrainz genres arrive free with the Lidarr index and are
+no use for this: they are sparse and formal, and will not tell you a record is
+screamo or synthwave. People typing tags will.
+
+Tags are fetched once per artist — one `artist.getTopTags` call each — and
+refreshed every ninety days. `--tag-limit` spreads the first sweep over several
+passes.
+
+Artists are looked up **by name**, not by MusicBrainz id, despite Lidarr having
+an id for every one of them. Last.fm's mbid index is stale and partial: it
+answers "the artist you supplied could not be found" for Devo, Escape the Fate,
+Blasterjaxx and a few hundred others whose pages plainly exist and carry exactly
+the tags wanted. Its name index is the one its own site runs on. The id is kept
+only as a fallback, for a name Lidarr spells differently.
+
+An artist neither key resolves is recorded as fetched with no tags, so the next
+pass does not spend a request on it again. A genuine failure — a rate limit, a
+bad key — is *not* recorded, so that one is retried.
+
+The built-in moods are built from the **tag distribution of this library**,
+measured, rather than from a general taxonomy:
+
+| Mood | Selected on |
+| --------------- | -------------------------------------------------------- |
+| `drum-and-bass` | drum and bass and its six spellings, liquid funk, neurofunk, jungle, techstep, hospital records |
+| `bass` | dubstep, brostep, grime, trip-hop, big beat |
+| `dance` | house and its variants, trance, techno, electro, rave |
+| `pop-punk` | pop punk, punk, emo, emocore, easycore, power pop |
+| `screamo` | screamo, post-hardcore, metalcore, melodic hardcore, trancecore |
+| `heavy-metal` | heavy metal, thrash, speed, power, death, prog, NWOBHM |
+| `hair-metal` | hair metal, glam metal, glam rock, arena rock, AOR — 1975-1994 |
+| `nu-metal` | nu metal, alternative metal, rapcore, industrial |
+| `classic-rock` | classic rock, prog, psychedelic, blues rock, 70s, 60s |
+| `80s-synths` | 80s, new wave, synth pop, electropop, post-punk — 1975-1992, rock excluded |
+| `indie` | indie, indie rock, indie pop, britpop, singer-songwriter |
+
+Measuring first mattered. `synthwave`, `edm`, `big room` and `hardstyle` are
+plausible tags that carry **nothing at all** here, while `techstep`, `easycore`
+and `hospital records` carry real weight. Guessing produces the first list.
+
+Three kinds of tag are never used, and there is a test enforcing it:
+
+- **Nationality** — `american` alone spans 241 artists. A passport is not a
+  sound.
+- **`rock` and `electronic`** — 340 and 275 artists, most of the library. A
+  mood that matches everything is not a mood.
+- **Artist names** — Last.fm's most popular tag for an artist is frequently
+  their own name. `green day`, `paramore` and `queen` are single-artist
+  playlists waiting to happen.
+
+### Exclusions
+
+A mood may also list `exclude`. An excluded tag drops the artist outright rather
+than docking their score, and it exists because `80s-synths` cannot be written
+any other way.
+
+`80s` is the eleventh most-played tag here, and it sits on Def Leppard and Bon
+Jovi exactly as heavily as on Eurythmics. Weighting cannot separate them,
+because the tag it would weight is the one they share. What does separate them
+is that the stadium rock also carries `hard rock` and `hair metal`, and the
+synth acts do not.
+
+Checked against live Last.fm pages, since the tag census only sees artists
+already in the library:
+
+| Artist | Tags |
+| --- | --- |
+| Eurythmics | `80s`, `new wave`, `pop`, `female vocalists`, `synth pop` |
+| Frankie Goes to Hollywood | `80s`, `new wave`, `pop`, `british`, `dance` |
+| Depeche Mode | `electronic`, `synthpop`, `new wave`, `80s`, `synth pop` |
+| Duran Duran | `new wave`, `80s`, `pop`, `synth pop`, `rock` |
+
+Four of Eurythmics' five tags are ones no mood may use. Three of the four
+artists spell it **`synth pop`** with a space; only one spells it `synthpop`.
+Guessing one spelling would have missed most of the canon.
+
+An artist qualifies when their tag weights inside a mood sum to at least 30 out
+of Last.fm's 0-100 scale. One low-weight tag is not a genre, it is somebody's
+stray opinion.
+
+`years` filters on the album's release date, which is what separates eighties
+synth records from everything else a synthpop tag drags in.
+
+`--vibes` replaces the whole set with a JSON file of the same shape, so a new
+mood does not need a new release:
+
+```json
+[{ "name": "shoegaze", "tags": ["shoegaze", "dream pop"], "min_score": 40 }]
+```
+
+Names are validated when the file is read, not when the file is written. A bad
+one would otherwise surface as a playlist created somewhere unintended.
+
+### Paths
+
+Lidarr knows where the lossless source is; the playlists have to point at the
+MP3s music-mirror made from it. The mapping strips a library root from Lidarr's
+track paths and re-roots them under the mirror, with the suffix changed.
+
+`--library-root` is derived from the common parent of the indexed artist folders
+when unset, so it agrees with Lidarr by construction rather than by being kept
+in step by hand. Override it if that guess is wrong.
+
+Entries are written **relative to the playlist file**, so one playlist works
+from the NAS, from a Mac over SMB, and from Linux, without rewriting.
+
+Each playlist is given the **owner and group of the mirror** it is written
+into. The image runs as root by default so that a bind mount of any ownership
+stays writable, and the cost of that is output owned by root — which the account
+serving the share cannot read, group bit or no group bit, because the group is
+also root. Copying the mirror's own ownership avoids having to be told what it
+should be, and does nothing when the two already agree.
+
+A track is only listed once its mirror file has been confirmed to exist. Lidarr
+holding the FLAC says nothing about whether the MP3 has been encoded yet. If a
+large number are missing, the run says so — that is what a wrong `--library-root`
+or `--mirror` looks like, since the paths then map to nothing at all.
+
+`_playlists/` survives music-mirror's prune: it only deletes `*.mp3`, and its
+empty-directory sweep skips a directory holding M3Us.
+
+## The cold report
+
+Which albums have files, have never had a single track played in the whole
+history, and have sat there long enough to have had the chance. Ranked by the
+disk they occupy, because that is the point of the exercise.
+
+**Album-level, not track-level.** A record with two played tracks is a record
+that gets played; picking the other ten off it leaves gaps rather than
+reclaiming anything worth having.
+
+The age floor is measured from the newest file in the album, not from the
+release date — what matters is how long it has been available to play, not how
+old the record is. `--cold-after` sets it, defaulting to a year.
+
+Artists whose *every* album is cold are counted separately. That is a different
+proposition from one cold record by somebody otherwise played, and Lidarr can
+only tag at artist level anyway.
+
+### What stops it
+
+The report refuses to produce anything at all when:
+
+- the scrobble backfill is unfinished
+- any artist failed to index, or any artist has no albums indexed
+- the library has not been indexed, or there is no history to judge against
+
+Each of those makes played music look unplayed, which is the single failure that
+costs a library. They are checked rather than trusted, because the report they
+gate is the one that ends in deletion.
+
+**Nothing is written to Lidarr.** Every call there is still a `GET`. Unmonitoring
+comes once the list has been looked at, because no flag protects against a list
+that is wrong.
 
 ## How the ingest works
 
@@ -76,6 +369,15 @@ music-curator --report-only   # report on the store, fetch nothing
 | `--interval` | `MUSIC_CURATOR_INTERVAL` | unset | Repeat forever, e.g. `45m`, `6h`, `1d` |
 | `--request-delay` | `MUSIC_CURATOR_REQUEST_DELAY` | `0.25` | Seconds between API requests |
 | `--backfill-limit` | `MUSIC_CURATOR_BACKFILL_LIMIT` | `0` | Cap backfill requests per pass; 0 for no cap |
+| `--lidarr-url` | `MUSIC_CURATOR_LIDARR_URL` | unset | Lidarr base URL, e.g. `http://lidarr:8686` |
+| `--lidarr-api-key` | `MUSIC_CURATOR_LIDARR_API_KEY` | unset | Lidarr API key |
+| `--mirror` | `MUSIC_CURATOR_MIRROR` | unset | Root of the MP3 mirror; playlists go here |
+| `--library-root` | `MUSIC_CURATOR_LIBRARY_ROOT` | derived | Prefix to strip from Lidarr's paths |
+| `--playlist-limit` | `MUSIC_CURATOR_PLAYLIST_LIMIT` | `100` | Most tracks in any one playlist |
+| `--vibes` | `MUSIC_CURATOR_VIBES` | built-in | JSON file of mood definitions |
+| `--tag-limit` | `MUSIC_CURATOR_TAG_LIMIT` | `0` | Cap artist tag lookups per pass |
+| `--cold-after` | `MUSIC_CURATOR_COLD_AFTER` | `365` | Days a file must sit unplayed to count as cold |
+| `--skip-index` | — | off | Match against the index already held |
 | `--report-only` | — | off | Report without fetching |
 
 A [Last.fm API key](https://www.last.fm/api/account/create) is all that is
@@ -107,9 +409,11 @@ docker build --target test .   # what CI runs
 pytest                         # needs pytest on PATH
 ```
 
-The suite runs against a fake transport that reproduces the real service's
-paging, its `from`/`to` semantics and its awkward response shapes. No network,
-no credentials, no rate limit. On a Nix machine:
+The suite runs against fake transports for both services. The Last.fm one
+reproduces its paging, its `from`/`to` semantics and its awkward response
+shapes; the Lidarr one serves a canned library split across the same four
+endpoints the indexer calls, so the stitching is exercised rather than
+stubbed. No network, no credentials, no rate limit. On a Nix machine:
 
 ```sh
 nix shell nixpkgs#python3Packages.pytest -c pytest
@@ -120,9 +424,11 @@ nix shell nixpkgs#python3Packages.pytest -c pytest
 | Stage | Status |
 | ------------------------------------------------ | ------------ |
 | Last.fm ingest and store | done |
-| Lidarr index and the scrobble-to-track matcher | next |
-| M3U playlists written into the mirror | after that |
-| Cold-music report, unmonitoring what is not played | last |
+| Lidarr index and the scrobble-to-track matcher | done |
+| M3U playlists from the listening history | done |
+| Genre and mood playlists from Last.fm tags | done |
+| Cold-music report | done |
+| Unmonitoring what is not played | last |
 
 The cull will unmonitor cold albums in Lidarr and tag their artists. It will
 never delete files: Lidarr's `AlbumResource` has no tags at all, so tagging

compose.yaml

@@ -17,6 +17,10 @@ services:
       # enough, none of the endpoints used here authenticate a user.
       MUSIC_CURATOR_LASTFM_API_KEY: set-me-in-the-truenas-ui
       MUSIC_CURATOR_DB: /data/curator.db
+      # Lidarr, for indexing the library. Read-only: every call is a GET.
+      # Leave unset to ingest scrobbles and nothing else.
+      MUSIC_CURATOR_LIDARR_URL: http://lidarr:8686
+      MUSIC_CURATOR_LIDARR_API_KEY: set-me-in-the-truenas-ui
       # How long to wait between passes. Each one catches up on new scrobbles
       # and continues the backfill if it has not finished.
       MUSIC_CURATOR_INTERVAL: 6h
@@ -26,5 +30,11 @@ services:
       # Cap the backfill at this many requests per pass. Unlimited by default,
       # which finishes a long history in one go.
       # MUSIC_CURATOR_BACKFILL_LIMIT: "0"
+      # The MP3 mirror music-mirror maintains. Playlists are written into
+      # _playlists/ inside it; leave unset to skip them.
+      MUSIC_CURATOR_MIRROR: /mirror
+      # Most tracks in any one playlist.
+      # MUSIC_CURATOR_PLAYLIST_LIMIT: "100"
     volumes:
       - /mnt/tank/apps/music-curator:/data
+      - /mnt/tank/media/music-mp3:/mirror

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "music-curator"
-version = "0.1.0"
+version = "0.6.0"
 description = "Ingest a Last.fm listening history and curate a music library from it"
 readme = "README.md"
 requires-python = ">=3.11"

tests/conftest.py

@@ -1,6 +1,10 @@
+import http.server
+import io
 import json
 import os
 import sys
+import threading
+import urllib.error
 import urllib.parse
 
 import pytest
@@ -52,7 +56,9 @@ class FakeLastfm:
     is prepended to the first page with no `date`.
     """
 
-    def __init__(self, tracks=(), loved=(), nowplaying=None, outcomes=()):
+    def __init__(self, tracks=(), loved=(), nowplaying=None, outcomes=(), tags=None):
+        # Keyed by mbid or by artist name, whichever the caller asked with.
+        self.tags = tags or {}
         self.tracks = sorted(tracks, key=lambda track: int(track["date"]["uts"]), reverse=True)
         self.loved = list(loved)
         self.nowplaying = nowplaying
@@ -78,6 +84,15 @@ class FakeLastfm:
             return json.dumps(self._recent(query))
         if method == "user.getlovedtracks":
             return json.dumps(self._loved(query))
+        if method == "artist.gettoptags":
+            key = query.get("mbid") or query.get("artist", "")
+            if key not in self.tags:
+                # What the real service says for a key it cannot resolve, which
+                # for mbids is a great many artists whose pages plainly exist.
+                return json.dumps(
+                    {"error": 6, "message": "The artist you supplied could not be found"}
+                )
+            return json.dumps({"toptags": {"tag": self.tags[key]}})
         raise AssertionError(f"unexpected method {method}")
 
     def _recent(self, query):
@@ -120,6 +135,115 @@ class FakeLastfm:
         }
 
 
+class FakeLidarr:
+    """A transport serving a canned library over the Lidarr v1 API surface.
+
+    Built from a nested description -- artist, album, tracks -- and split back
+    out across the four endpoints the indexer actually calls, so the tests
+    exercise the same stitching the real thing does.
+    """
+
+    def __init__(self, artists=(), fail=()):
+        # (path, artistId) pairs the fake refuses to serve, standing in for the
+        # 500s Lidarr returns on data it cannot hydrate.
+        self.fail = set(fail)
+        self.artists, self.albums, self.tracks, self.files = [], [], [], []
+        for artist_index, entry in enumerate(artists, start=1):
+            artist_id = artist_index
+            self.artists.append(
+                {
+                    "id": artist_id,
+                    "artistName": entry["name"],
+                    "foreignArtistId": entry.get("mbid", f"artist-mbid-{artist_id}"),
+                    "path": f"/music/{entry['name']}",
+                    "monitored": entry.get("monitored", True),
+                }
+            )
+            for album_index, album in enumerate(entry.get("albums", []), start=1):
+                album_id = artist_id * 100 + album_index
+                self.albums.append(
+                    {
+                        "id": album_id,
+                        "artistId": artist_id,
+                        "title": album["title"],
+                        "foreignAlbumId": f"album-mbid-{album_id}",
+                        "monitored": album.get("monitored", True),
+                        "releaseDate": album.get("release_date", "2019-01-01T00:00:00Z"),
+                    }
+                )
+                for track_index, track in enumerate(album.get("tracks", []), start=1):
+                    track_id = album_id * 100 + track_index
+                    has_file = track.get("has_file", True)
+                    self.tracks.append(
+                        {
+                            "id": track_id,
+                            "artistId": artist_id,
+                            "albumId": album_id,
+                            "title": track["title"],
+                            "foreignRecordingId": track.get("recording_mbid", ""),
+                            "trackFileId": track_id if has_file else 0,
+                            "hasFile": has_file,
+                            "duration": 210000,
+                        }
+                    )
+                    if has_file:
+                        self.files.append(
+                            {
+                                "id": track_id,
+                                "artistId": artist_id,
+                                "albumId": album_id,
+                                "path": f"/music/{entry['name']}/{album['title']}/"
+                                f"{track['title']}.flac",
+                                "dateAdded": track.get("added", "2020-05-01T12:00:00Z"),
+                            }
+                        )
+        self.calls = []
+
+    def __call__(self, url, timeout=None, headers=None):
+        parsed = urllib.parse.urlparse(url)
+        assert (headers or {}).get("X-Api-Key"), "Lidarr requires the API key header"
+        path = parsed.path.rsplit("/", 1)[-1]
+        query = {
+            key: value[0] for key, value in urllib.parse.parse_qs(parsed.query).items()
+        }
+        self.calls.append((path, query))
+
+        artist_id = int(query.get("artistId", 0))
+        if (path, artist_id) in self.fail:
+            raise urllib.error.HTTPError(
+                url, 500, "Internal Server Error", {}, io.BytesIO(b'{"message": "boom"}')
+            )
+
+        album_ids = query.get("albumIds")
+        if path == "album" and album_ids:
+            album_id = int(album_ids)
+            if ("albumid", album_id) in self.fail:
+                raise urllib.error.HTTPError(
+                    url,
+                    500,
+                    "Internal Server Error",
+                    {},
+                    io.BytesIO(b'{"message": "Sequence contains more than one element"}'),
+                )
+            return json.dumps([row for row in self.albums if row["id"] == album_id])
+
+        if path == "artist":
+            return json.dumps(self.artists)
+        source = {"album": self.albums, "track": self.tracks, "trackfile": self.files}[path]
+        if path == "album" and not artist_id:
+            # Lidarr's unfiltered album endpoint returns the lot.
+            return json.dumps(source)
+        if not artist_id:
+            raise urllib.error.HTTPError(
+                url,
+                400,
+                "Bad Request",
+                {},
+                io.BytesIO(b'{"message": "artistId must be provided"}'),
+            )
+        return json.dumps([row for row in source if row["artistId"] == artist_id])
+
+
 @pytest.fixture
 def now_playing():
     """The entry Last.fm prepends for a track in progress: no `date` at all."""
@@ -130,3 +254,53 @@ def now_playing():
         "mbid": "",
         "@attr": {"nowplaying": "true"},
     }
+
+
+class _CountingServer(http.server.ThreadingHTTPServer):
+    """Counts accepted connections, which is what connection reuse is about."""
+
+    daemon_threads = True
+
+    def __init__(self, *args, **kwargs):
+        self.connections = 0
+        super().__init__(*args, **kwargs)
+
+    def process_request(self, request, client_address):
+        self.connections += 1
+        super().process_request(request, client_address)
+
+
+class _Handler(http.server.BaseHTTPRequestHandler):
+    # Without HTTP/1.1 the server closes after every response and no client
+    # could reuse anything, which would make the test prove nothing.
+    protocol_version = "HTTP/1.1"
+
+    def do_GET(self):
+        if self.path.startswith("/boom"):
+            body, status = b'{"message": "boom"}', 500
+        else:
+            body = json.dumps(
+                {"path": self.path, "key": self.headers.get("X-Api-Key")}
+            ).encode()
+            status = 200
+        self.send_response(status)
+        self.send_header("Content-Type", "application/json")
+        self.send_header("Content-Length", str(len(body)))
+        self.end_headers()
+        self.wfile.write(body)
+
+    def log_message(self, *args):
+        pass
+
+
+@pytest.fixture
+def http_server():
+    """A real local HTTP server, for the one component that talks sockets."""
+    server = _CountingServer(("127.0.0.1", 0), _Handler)
+    thread = threading.Thread(target=server.serve_forever, daemon=True)
+    thread.start()
+    try:
+        yield server, f"http://127.0.0.1:{server.server_port}"
+    finally:
+        server.shutdown()
+        server.server_close()