test: prove the database lands at the device root, not in the music folder #10

2026-08-26T13:31:40+01:00

lyrathorpe commented

2026-08-26 13:31:40 +01:00

The problem this proves is handled

The two tools disagree about where the root is.


`destination`	`/media/IPOD/Music` — where the artist folders go
`mounted_on`	`/media/IPOD` — derived, the device root
`device_prefix`	`/Music` — the difference

rsync operates at the Music level. The database builder must run one level
up, where .rockbox lives, and must record /Music/... paths while reading
the bytes from the mirror. device_prefix reconciles them, by building a
scratch root that looks like the device:

~/.cache/music-mirror/database/
├── .rockbox/          <- real; the .tcd files are written here
└── Music -> /mnt/tank/media/music-mp3

Until now that was argued rather than demonstrated — the earlier tests could
not create a mount point, so findmnt --target resolved to the host's real
root and the step skipped itself.

The test

Unprivileged user namespaces allow a genuine bind mount, so this creates an
actual mount point and exercises the derivation rather than grepping the
script. A stub builder records its working directory and what it could see.

Asserted:

the .tcd file arrives beside .rockbox, not inside Music/
nothing else is written to the device — only *.tcd is copied, so the stub's
own markers stay in the scratch
the build ran in the scratch root, not on the card
it could walk into the mirror through the symlink, which is the mechanism
producing device paths from mirror bytes

Skips where user namespaces are unavailable, which includes the CI container:
131 passed, 1 skipped there; 132 passed locally.

## The problem this proves is handled The two tools disagree about where the root is. | | | | --- | --- | | `destination` | `/media/IPOD/Music` — where the artist folders go | | `mounted_on` | `/media/IPOD` — derived, the device root | | `device_prefix` | `/Music` — the difference | rsync operates at the `Music` level. The database builder must run one level up, where `.rockbox` lives, and must record `/Music/...` paths *while reading the bytes from the mirror*. `device_prefix` reconciles them, by building a scratch root that looks like the device: ``` ~/.cache/music-mirror/database/ ├── .rockbox/ <- real; the .tcd files are written here └── Music -> /mnt/tank/media/music-mp3 ``` Until now that was argued rather than demonstrated — the earlier tests could not create a mount point, so `findmnt --target` resolved to the host's real root and the step skipped itself. ## The test Unprivileged user namespaces allow a genuine bind mount, so this creates an actual mount point and exercises the derivation rather than grepping the script. A stub builder records its working directory and what it could see. Asserted: - the `.tcd` file arrives beside `.rockbox`, **not** inside `Music/` - nothing else is written to the device — only `*.tcd` is copied, so the stub's own markers stay in the scratch - the build ran in the scratch root, not on the card - it could walk *into* the mirror through the symlink, which is the mechanism producing device paths from mirror bytes Skips where user namespaces are unavailable, which includes the CI container: 131 passed, 1 skipped there; 132 passed locally.

lyrathorpe added 1 commit 2026-08-26 13:31:40 +01:00

test: prove the database lands at the device root, not in the music folder

Build and publish container / build (pull_request) Successful in 4m30s

Details

633fbbaf91

The two tools disagree about where the root is. rsync copies artist folders
into <device>/Music, while the database builder must run one level up, where
.rockbox lives, and must record /Music/... paths despite reading the bytes from
the mirror. device_prefix is what reconciles them, and until now that was only
argued rather than demonstrated.

Unprivileged user namespaces make a real bind mount possible, so the test can
create an actual mount point and exercise the derivation instead of asserting
the shape of the script. A stub builder records its working directory and what
it could see. The test asserts the .tcd file arrives beside .rockbox rather
than inside Music, that the build ran in the scratch root and not on the card,
and that it could walk into the mirror through the symlink -- which is the
mechanism that produces device paths from mirror bytes.

It skips where user namespaces are unavailable, which includes the CI
container.

lyrathorpe added 1 commit 2026-08-26 13:58:48 +01:00

fix: stop counting by default; the pass costs more than the transfer

Build and publish container / build (pull_request) Successful in 3m32s

Details

19ac9e5d92

The counting pass was added so the progress line could show a percentage and an
estimate, and on a real card it turned out to dominate the run. Measured
against the device: reading a track from the SMB mirror ran at 35 MB/s and
writing to the card at 21 MB/s, while the sync itself managed tens of kilobytes
per second. Neither end was slow. The cost was traversing fifty thousand files
across six thousand directories on FAT, and the counting pass does that a
second time, comparing both trees in full exactly as the transfer does.

Counting is now opt-in behind -P. Without it the progress line still shows the
running count, the transfer rate and the album in flight; the percentage and
the estimate are what needed the extra walk, and they were the least useful
part of the display.

That the fix for "it looks hung" was itself making it slow is the sort of thing
only measuring catches. The line still answers the question it was added for --
whether anything is happening -- without paying for the part that merely made
it prettier.

lyrathorpe added 1 commit 2026-08-26 18:25:06 +01:00

feat: convert Rockbox's playback log on the laptop, skipping the plugin

Build and publish container / build (pull_request) Successful in 5m6s

Details

f324b1b720

Scrobbling previously needed the on-device Last.fm plugin run by hand before
each sync, to turn Rockbox's playback log into AUDIOSCROBBLER format. Forgetting
that step means the sync submits nothing and quietly appears not to work.

Core Rockbox writes ROCKBOX_DIR/playback.log whenever "play log" is enabled,
with no plugin running at all. Each line is timestamp:elapsed_ms:length_ms:path.
The only thing missing is tags, and that is exactly why the plugin exists:
reading them back off the player is slow. Off the mirror it is free, because the
same files are already there -- so the conversion belongs on the laptop, and the
plugin can be skipped entirely.

A play counts as listened at half the track's length, matching the plugin's
savepct default, so the two cannot disagree about what a play was. A short play
is a skip. An entry with no usable timestamp is refused rather than invented,
which is the clockless case again. A path that maps to nothing in the mirror is
counted and reported instead of guessed at.

Rotated logs are picked up too; Rockbox starts a new one past half a megabyte.
All of them are renamed aside together once Last.fm has accepted the batch.

A .scrobbler.log is still read when the plugin has been run and left one.

lyrathorpe added 1 commit 2026-08-26 18:33:05 +01:00

fix: never discard a play that has not been submitted

Build and publish container / build (pull_request) Successful in 5m11s

Details

374a17474f

Setting the logs aside after a successful submission threw away more than it
had submitted. A play of a track missing from the mirror -- not yet copied, or
its tags unreadable -- was counted as unresolved and then carried off with the
rest, with nothing to retry it. The play happened and was lost.

Two obligations now, kept separate. The original log is renamed rather than
deleted, so a mistake here cannot destroy the record. And every play that was
not submitted is written back into a live log, so the next run attempts it
again: the unmatched ones, and anything in a batch that failed.

Submission is recorded batch by batch as each is accepted, so a failure partway
through knows exactly what got through. The remainder is written back and
nothing is sent twice. When nothing at all is accepted the logs are left
untouched.

Skips and clockless entries are deliberately not retained. Neither can ever be
submitted, so keeping them would mean reprocessing them for ever, and the
untouched original holds them regardless.

Also fixes a way to lose the lot: read_tags caught OSError and ValueError, but
ffprobe failing raises CalledProcessError, which is neither. A single unreadable
file aborted the whole submission rather than costing one unidentified play.

lyrathorpe merged commit 8d6885c46a into main

2026-08-26 20:38:53 +01:00

lyrathorpe deleted branch feat/database-in-sync

2026-08-26 20:38:54 +01:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: lyrathorpe/music-mirror#10