fix: hold one connection to Lidarr open, and retry what deserves retrying #4

2026-08-24T14:40:45+01:00

lyrathorpe commented

2026-08-24 14:40:45 +01:00

Problem

WARNING could not fetch albums for Twin Atlantic: <urlopen error [Errno -3] Try again>
WARNING could not fetch albums for A Tribe Called Quest: <urlopen error [Errno -3] Try again>

Errno -3 is EAI_AGAIN: DNS resolution failing. It was failing for everything,
and the cause is how this client makes requests rather than anything wrong with
Lidarr.

Indexing is two requests per artist, and more once the album fallback fires. On
a large library that is thousands of requests in a few minutes, and urllib
opens a new TCP connection -- and performs a new DNS lookup -- for every single
one. That is enough to exhaust a container's resolver.

Two mistakes compounded it:

No retries. Declined earlier on the grounds that "Lidarr is on the same
LAN". Not a safe assumption: it can sit behind a public hostname and a
reverse proxy. A momentary failure cost an artist their entire entry.
The album probe. Added in #3 to name the offending album. It costs one
request per album of the failing artist, which is right for a deterministic
fault and actively harmful during a network-wide one -- every artist fails,
and each one gets probed, multiplying the load responsible.

Change

A transport holding one connection open per host. The name is resolved once
and the socket is reused. It retries once on a connection the server has
already closed, since a stale keep-alive only announces itself on use.
Transient failures -- dropped connections, resolver hiccups, 429, 502,
503, 504 -- retried with a backoff.
HTTP 500 deliberately excluded from retries. It is an exception inside
Lidarr's serialisation, not a busy server, and three attempts only delay
finding that out.
That same transient/deterministic split gates the probe, so a network outage
is no longer investigated artist by artist.

Testing

53 tests, green locally and in docker build --target test.

The keep-alive transport is tested against a real local HTTP server, not a
fake: connection reuse and status mapping are precisely the properties a fake
would assume rather than demonstrate. Mutation-checked -- removing the reuse
makes the server see five connections instead of one, and the test fails.

Note on deployment

A local address is preferable to lidarr.emmaisvery.gay for this. It takes DNS,
the reverse proxy and its timeouts out of a path that needs none of them. The
fix is worth having either way.

## Problem WARNING could not fetch albums for Twin Atlantic: <urlopen error [Errno -3] Try again> WARNING could not fetch albums for A Tribe Called Quest: <urlopen error [Errno -3] Try again> Errno -3 is `EAI_AGAIN`: DNS resolution failing. It was failing for everything, and the cause is how this client makes requests rather than anything wrong with Lidarr. Indexing is two requests per artist, and more once the album fallback fires. On a large library that is thousands of requests in a few minutes, and `urllib` opens a new TCP connection -- and performs a new DNS lookup -- for every single one. That is enough to exhaust a container's resolver. Two mistakes compounded it: - **No retries.** Declined earlier on the grounds that "Lidarr is on the same LAN". Not a safe assumption: it can sit behind a public hostname and a reverse proxy. A momentary failure cost an artist their entire entry. - **The album probe.** Added in #3 to name the offending album. It costs one request per album of the failing artist, which is right for a deterministic fault and actively harmful during a network-wide one -- every artist fails, and each one gets probed, multiplying the load responsible. ## Change - A transport holding one connection open per host. The name is resolved once and the socket is reused. It retries once on a connection the server has already closed, since a stale keep-alive only announces itself on use. - Transient failures -- dropped connections, resolver hiccups, `429`, `502`, `503`, `504` -- retried with a backoff. - HTTP `500` deliberately excluded from retries. It is an exception inside Lidarr's serialisation, not a busy server, and three attempts only delay finding that out. - That same transient/deterministic split gates the probe, so a network outage is no longer investigated artist by artist. ## Testing 53 tests, green locally and in `docker build --target test`. The keep-alive transport is tested against a **real local HTTP server**, not a fake: connection reuse and status mapping are precisely the properties a fake would assume rather than demonstrate. Mutation-checked -- removing the reuse makes the server see five connections instead of one, and the test fails. ## Note on deployment A local address is preferable to `lidarr.emmaisvery.gay` for this. It takes DNS, the reverse proxy and its timeouts out of a path that needs none of them. The fix is worth having either way.

lyrathorpe added 1 commit 2026-08-24 14:40:45 +01:00

fix: hold one connection to Lidarr open, and retry what deserves retrying

Build and publish container / build (pull_request) Successful in 6m0s

Details

edebecc8ea

Indexing makes two requests per artist, and more when the album fallback fires.
urllib opens a new TCP connection and performs a new DNS lookup for every one of
them, so a large library becomes thousands of lookups inside a few minutes. That
is enough to exhaust a container's resolver, and the result is
"[Errno -3] Try again" on every artist at once -- a failure caused entirely by
how the requests were made rather than by anything wrong with Lidarr.

Add a transport that keeps one connection open per host, so the name is resolved
once and the socket is reused. It retries once on a connection the server has
already closed, since a stale keep-alive announces itself only on use.

Retries were previously declined on the grounds that Lidarr is on the same LAN.
That is not a safe assumption -- it may sit behind a public hostname and a
reverse proxy -- and a transient failure currently costs an artist their entire
entry for that pass. Transient failures are now retried with a backoff. HTTP 500
is deliberately excluded: it is an exception inside Lidarr's serialisation, not
a busy server, and three attempts only delay finding that out.

The same distinction gates the album probe added alongside this. Naming the
offending album costs one request per album of that artist, which is worth it
for a deterministic fault and actively harmful during a network-wide one, where
every artist fails and probing each of them multiplies the load responsible.

The keep-alive transport is tested against a real local HTTP server rather than
a fake, because connection reuse and status mapping are exactly the properties a
fake would assume rather than demonstrate.

lyrathorpe merged commit fef082a783 into main

2026-08-24 14:41:02 +01:00

lyrathorpe deleted branch fix/lidarr-connection-reuse

2026-08-24 14:41:03 +01:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: lyrathorpe/music-curator#4