6 Commits

Include renames

Author	SHA1	Message	Date
Emma Thorpe	40dfce8a4c	fix: look tags up by artist name rather than by MusicBrainz id ... Tag lookups asked by MusicBrainz id whenever Lidarr had one, which is always. The reasoning was that an id cannot be ambiguous the way a name can. In practice Last.fm's mbid index is stale and partial, and it answered "the artist you supplied could not be found" for Devo, Escape the Fate, Blasterjaxx, Frank Carter & the Rattlesnakes and a long tail of others -- artists whose Last.fm pages plainly exist and carry precisely the tags the moods are built from. Devo is tagged new wave, post-punk and 80s; Escape the Fate is tagged post-hardcore, screamo and emocore. Both were skipped. Ask by name first, which is the index Last.fm's own site runs on, and keep the id only as a fallback for a name Lidarr spells differently. Failures were also being dropped without recording the attempt, so every one of those artists was re-queried on every subsequent pass, indefinitely. They are now distinguished: an artist that neither key resolves is recorded as fetched with no tags and not asked about again, while a genuine failure -- a rate limit, a bad key -- is deliberately left unrecorded so the next pass retries it. Telling the two apart needed the service's own error number, so LastfmError now carries it.	2026-08-24 18:13:23 +01:00
Emma Thorpe	15e5ee5aea	feat: mood playlists from Last.fm crowd tags ... A second set of playlists selecting by genre and mood rather than by play history: eighties synths, high energy rock, screamo, drum and bass, dance, classic rock. The tags come from Last.fm rather than MusicBrainz. MusicBrainz genres arrive free with the Lidarr index, which makes them the obvious choice and the wrong one: they are sparse and formal, and will not tell you a record is screamo or synthwave. Crowd tags will, because people typed them. One request per artist, by MusicBrainz id where Lidarr has one, refreshed every ninety days. An artist Last.fm has never heard of is recorded as fetched with no tags rather than left unmarked, so it is not asked about again on every pass forever. The tag table is keyed on the normalised artist name, not the Lidarr id, so it survives an artist being removed and re-added there. Weights are taken from the response's count where it has one. The documented sample carries only a name and a URL, a live response also carries a 0-100 count, and depending on either alone would be a guess -- so the count is used when present and the documented ordering by popularity stands in when it is not. An artist qualifies for a mood when their weights inside it sum to at least thirty. A single low-weight tag is not a genre, it is somebody's stray opinion. A mood may also restrict release years, which is what separates eighties synth records from everything else a synthpop tag drags in. The built-in set is chosen for this library rather than as a taxonomy, and --vibes replaces it wholesale with a JSON file so a new mood does not need a new release. Names are validated when that file is read: an invalid one would otherwise only surface as a playlist written somewhere unintended.	2026-08-24 17:56:35 +01:00
Emma Thorpe	edebecc8ea	fix: hold one connection to Lidarr open, and retry what deserves retrying ... Indexing makes two requests per artist, and more when the album fallback fires. urllib opens a new TCP connection and performs a new DNS lookup for every one of them, so a large library becomes thousands of lookups inside a few minutes. That is enough to exhaust a container's resolver, and the result is "[Errno -3] Try again" on every artist at once -- a failure caused entirely by how the requests were made rather than by anything wrong with Lidarr. Add a transport that keeps one connection open per host, so the name is resolved once and the socket is reused. It retries once on a connection the server has already closed, since a stale keep-alive announces itself only on use. Retries were previously declined on the grounds that Lidarr is on the same LAN. That is not a safe assumption -- it may sit behind a public hostname and a reverse proxy -- and a transient failure currently costs an artist their entire entry for that pass. Transient failures are now retried with a backoff. HTTP 500 is deliberately excluded: it is an exception inside Lidarr's serialisation, not a busy server, and three attempts only delay finding that out. The same distinction gates the album probe added alongside this. Naming the offending album costs one request per album of that artist, which is worth it for a deterministic fault and actively harmful during a network-wide one, where every artist fails and probing each of them multiplies the load responsible. The keep-alive transport is tested against a real local HTTP server rather than a fake, because connection reuse and status mapping are exactly the properties a fake would assume rather than demonstrate.	2026-08-24 14:40:00 +01:00
Emma Thorpe	3e78f8ebd4	fix: index albums through the endpoint Lidarr does not throw from ... Indexing fetched albums one artist at a time, and `GET /api/v1/album?artistId=` is Lidarr's unguarded path. It maps straight from the album service with no hydration: the mapper then dereferences model.Images and model.SecondaryTypes without a null check, follows model.Artist?.Value where only the first link is guarded, and selects the monitored release with SingleOrDefault, which throws outright when an album has two of them. Any of those is a 500 that aborts the whole index. The unfiltered `GET /api/v1/album` builds its own artist and release lookups and skips an album whose metadata is missing rather than dereferencing it. Use that instead, once, and group by artistId locally. It is the defensive path and it costs N fewer requests. Tracks and files have no unfiltered endpoint -- Lidarr rejects a call with no filter at all -- so those stay per artist. A failure on one artist now skips that artist rather than ending the run, but the count is recorded in the store and the coverage report leads with it: a missing artist makes their played music look unplayed, which is precisely the error that costs music later, so an incomplete index must not be culled against. Errors now carry the request URL and whatever the server put in the body. The original report of this failure was "album: HTTP 500", which points at the URL and the credentials -- neither of which was at fault.	2026-08-24 14:05:21 +01:00
Emma Thorpe	5c4797ef38	feat: index the library from Lidarr and match it against the scrobbles ... Stage two. The scrobble history says what was played by name; Lidarr says what is owned, and where the files are. Neither is useful for curation until the two are tied together, and the quality of that join is what decides whether the later cull can be trusted at all. The index is a wholesale rebuild of every artist, album and track Lidarr holds, including file paths and the date each file landed -- the latter for the age floor a cull will need. It is rebuilt rather than reconciled because Lidarr is the authority and a deletion there has to disappear here, not linger as a library entry with no file behind it. Every call is a GET; nothing is written back. Matching runs at the level of the distinct artist/track pair rather than the individual play, because a verdict is a property of the name pair and there are three plays for every one of them. Two tiers: a MusicBrainz recording id, which Last.fm supplies per scrobble and Lidarr exposes as ForeignRecordingId, gives an exact join; everything else falls to a normalised name comparison. There is deliberately no third tier. A near-miss guess is worse than an admitted one, since the entire purpose of the resulting number is to state how far the matching can be relied on. Normalisation folds the ways the two sides habitually disagree: case, accents, guest credits that Last.fm puts in the artist field, trailing version suffixes, ampersands, and a leading article. Punctuation needs two opposing rules and both are load-bearing -- apostrophes are deleted so "Don't" meets "Dont", while every other mark becomes a space so "AC/DC", "AC-DC" and "AC DC" meet as well. It errs towards collapsing too much: a false match makes a track look played, a missed match makes it look abandoned, and only the second one loses music. The coverage report deliberately does not lead with matched versus unmatched. Most unmatched listening is music that was never in the library and says nothing about the matcher. The figure that matters is unmatched listening by an artist the library does hold: a track that was played, sitting next to a file it should have matched. The worst fifteen are listed by play count. The schema gains its tables additively and migrates a version 1 store in place, because rebuilding a nine-year history costs several thousand API requests.	2026-08-24 13:31:08 +01:00
Emma Thorpe	18f05d3d55	feat: ingest a Last.fm scrobble history into a local store ... First stage of a curation tool for the music library that music-mirror mirrors. Before anything can build playlists or decide what has gone cold, there has to be a local, queryable record of what is actually played; an API call per question does not scale to a library-sized analysis. Ingest is in two halves. A catch-up fetches everything scrobbled since the newest scrobble held, and a backfill walks the history backwards until it runs out. Both take their bounds from the database rather than from a saved cursor, so an interrupted run resumes from what it actually has, and both windows are bounded at each end so paging cannot shift under the fetch while new scrobbles arrive mid-run. Scrobbles carry no identifier, so the primary key is timestamp, artist and track. Two plays of one track in the same second collapse into a single row: they are indistinguishable in the data, and a surrogate key would make re-ingest non-idempotent, which is the worse trade. Three API behaviours are handled explicitly because each fails silently: the currently-playing track arrives with no timestamp and would be re-ingested on every pass; a lone result is returned as a bare object rather than a one-item list; and MBIDs are empty strings rather than absent when unknown, which would later look like a usable join key. Retries cover the rate limit and the transient backend errors with an exponential backoff. An invalid or suspended key fails immediately. The report exists to surface one number before the next stage is built: the share of scrobbles carrying a MusicBrainz recording id. Lidarr exposes the same identifier per track, so those can be joined exactly and the rest must go through name matching. That percentage bounds how far the matcher can be trusted. No runtime dependencies, and the tests run against a fake transport that reproduces the service's paging and response shapes, so they need neither network nor credentials.	2026-08-24 12:03:08 +01:00