The two tools disagree about where the root is. rsync copies artist folders
into <device>/Music, while the database builder must run one level up, where
.rockbox lives, and must record /Music/... paths despite reading the bytes from
the mirror. device_prefix is what reconciles them, and until now that was only
argued rather than demonstrated.
Unprivileged user namespaces make a real bind mount possible, so the test can
create an actual mount point and exercise the derivation instead of asserting
the shape of the script. A stub builder records its working directory and what
it could see. The test asserts the .tcd file arrives beside .rockbox rather
than inside Music, that the build ran in the scratch root and not on the card,
and that it could walk into the mirror through the symlink -- which is the
mechanism that produces device paths from mirror bytes.
It skips where user namespaces are unavailable, which includes the CI
container.
The on-device database commit does not work at this library size. It sorts the
whole index in whatever memory core_alloc_maximum() can scrape together, and on
fifty thousand tracks it runs for hours or aborts with a data abort -- observed
across several builds including stable.
Rockbox ships a host-side builder for exactly this, and the sync is the moment
the library changes, so it belongs here. MUSIC_MIRROR_DATABASE_TOOL points at
it; the step is skipped with a note when unset, as the scrobbler step is.
The scan runs against a scratch root -- a real .rockbox beside a symlink
standing in for wherever the music lands on the device -- so the paths recorded
are the ones Rockbox will look up, while the bytes are read from the mirror
rather than over USB. Only the dozen .tcd files cross to the card. Verified:
scanning through the symlink records /Music/... paths while reading from
somewhere else entirely.
The scratch root is kept between runs because the builder is incremental. A
second pass over unchanged files performs no metadata reads and finishes in a
fraction of a second, so only the first build pays the full cost.
That cost, measured rather than guessed: about 49 reads and 43 seeks per file,
the parser probing the head for ID3v2 and the tail for ID3v1. Two thousand
files in half a second on local disk. Over SMB the opens and the head/tail
split are real round trips, making a first full scan minutes rather than
seconds -- still preferable to an on-device commit that does not finish. There
is nothing to parallelise: the tool is single-threaded and two instances cannot
produce one database.
rsync itself is safe under interruption. It writes to a hidden temporary file
and renames it into place only once complete, and by default deletes any
partial file when interrupted -- verified both in the manual and by killing a
transfer and inspecting what was left, which was nothing. --partial is
deliberately absent and there is now a test asserting it stays that way. An
unclean kill can leave a hidden .track.mp3.XXXXXX behind; it is unplayable, it
is not in the source, and the next run's --delete removes it.
The script was not safe. Ctrl-C killed it before the sync and the unmount,
leaving a journal-less FAT filesystem holding dirty buffers -- which is the
exact corruption the script exists to prevent, arrived at by the most likely
route a person would take.
INT and TERM are now trapped. Both the normal path and the interrupt path call
the same finish function, so the flush and the unmount cannot drift apart, and
an interrupted run exits 130 rather than pretending to have succeeded.
The test is structural rather than timed. Reproducing a mid-transfer signal
needs a payload large enough to be slow, and a test that depends on winning a
race is a test that fails in CI for reasons that have nothing to do with the
code. The behaviour was verified by hand: SIGTERM mid-transfer gave exit 130,
the flush ran, and the destination held no short files and no leftover
temporaries.
The transfer is metadata-bound rather than throughput-bound. Fifty thousand
files is fifty thousand round trips, and the counting pass added for the
percentage doubles that.
--whole-file is already implied when both ends are local paths, which an SMB or
FAT mount is, but stating it records that the delta algorithm is deliberately
unwanted here: it would read every destination file back over USB to checksum
it, in order to avoid resending an MP3 that has changed in its entirety anyway.
--omit-dir-times drops one setattr per directory. Across six thousand album
folders on a FAT card that is six thousand operations spent on timestamps
nothing reads.
-Q skips the counting pass. The percentage and the estimate are worth a second
walk of a local tree and frequently are not worth one of a network mount, so
that is now a choice rather than a fixed cost.
The README covers the part that is not an rsync flag at all: SMB defaults to a
one second attribute cache, so nearly every stat goes to the wire, twice. An
actimeo of sixty on the mount does more than any of the above, and closes most
of the gap that would otherwise argue for moving to NFS.
The script required the destination to be its own mount point, while the
documentation and its own usage text both told the user to pass
/media/IPOD/Music. The documented invocation was rejected.
A subdirectory is the better target, so the guard was what was wrong. --delete
is confined to it, and the device path budget is now derived from it -- the
part of the destination below its mount point -- rather than configured, so the
budget cannot disagree with where the files are actually going. The check that
matters is that the destination sits on a FAT filesystem, which is also what
catches an unmounted device: /media/IPOD/Music then resolves to the host's own
root filesystem, and emptying that is the outcome all of these guards exist to
prevent.
Three further faults found by testing the guards rather than reasoning about
them:
Stripping the trailing slash from "/" left an empty string, so the guard
refusing the host root never fired and the user got "destination is not a
directory" instead.
die() printed only its first argument, so the second half of the non-FAT
message -- the half saying to check whether the device is mounted -- was
silently dropped.
--help was not handled at all. Only -h reached the usage text, and it exited 2
to stderr, which is right for misuse and wrong for someone asking a question.
Help now goes to stdout and exits zero, and carries the guidance rather than
leaving it to the README, since the question it answers is asked at a terminal.
The test stage installs bash, rsync and findmnt, none of which are in the base
image, and the tests skip rather than fail where they are absent -- a machine
without rsync is not a machine that would run this script.
Emma Thorpe