rsync reports each file's size with %l as it completes, which is all an
estimate needs: bytes done over time elapsed is the same arithmetic rsync would
do internally, and requires nothing it does not already print. The scan pass now
sums those sizes as well as counting files, so both a percentage and an estimate
have a real denominator.
The rate is measured over a trailing thirty seconds rather than the whole run,
so it follows a device that slows down instead of averaging the slowdown away --
which for a card reader that thermally throttles, or a USB link that renegotiates
after an hour, is the difference between a useful estimate and a reassuring one.
Below two seconds no rate is reported at all. The first handful of files arrive
microseconds apart, and dividing by that window produces a rate in the gigabytes
per second and an estimate of zero, which is worse than showing nothing.
Directory entries are excluded from the byte total as well as the file count.
rsync reports them with a 4096 inode size, which across six thousand album
directories is several megabytes of transfer that never happens.
The transfer looked hung. rsync prints nothing while it builds its file list,
which on fifty thousand files over USB is several minutes of silence, and
--info=progress2 does not help: with incremental recursion its percentage is
computed against a list rsync has not finished discovering, so it moves
backwards as often as forwards.
The script now counts what needs copying first and says so, then renders its
own single line that rewrites in place, showing the album currently going
across and a percentage against a total that is actually known. Counting costs
a second pass over the tree. That is the price of a percentage meaning
something, and it is cheaper than staring at a blank terminal wondering whether
the thing has died.
Directories are excluded from the count. rsync reports those too, and including
them puts the figure past a hundred per cent.
Piped to a log the line becomes a plain one every thirty seconds, because a log
full of carriage returns and escape codes is not a log anybody reads.
The script required the destination to be its own mount point, while the
documentation and its own usage text both told the user to pass
/media/IPOD/Music. The documented invocation was rejected.
A subdirectory is the better target, so the guard was what was wrong. --delete
is confined to it, and the device path budget is now derived from it -- the
part of the destination below its mount point -- rather than configured, so the
budget cannot disagree with where the files are actually going. The check that
matters is that the destination sits on a FAT filesystem, which is also what
catches an unmounted device: /media/IPOD/Music then resolves to the host's own
root filesystem, and emptying that is the outcome all of these guards exist to
prevent.
Three further faults found by testing the guards rather than reasoning about
them:
Stripping the trailing slash from "/" left an empty string, so the guard
refusing the host root never fired and the user got "destination is not a
directory" instead.
die() printed only its first argument, so the second half of the non-FAT
message -- the half saying to check whether the device is mounted -- was
silently dropped.
--help was not handled at all. Only -h reached the usage text, and it exited 2
to stderr, which is right for misuse and wrong for someone asking a question.
Help now goes to stdout and exits zero, and carries the guidance rather than
leaving it to the README, since the question it answers is asked at a terminal.
The test stage installs bash, rsync and findmnt, none of which are in the base
image, and the tests skip rather than fail where they are absent -- a machine
without rsync is not a machine that would run this script.
tools/sync-to-ipod.sh does the whole transfer to a Rockbox device, so the only
manual part left is the disk-mode button sequence.
The guards are the substance rather than decoration. rsync --delete is being
aimed at a whole filesystem, so the destination must exist, be its own mount
point, and be a FAT filesystem; the mirror must be non-empty and must not be
the destination. Emptying the wrong directory is not a mistake that announces
itself.
It also excludes /.rockbox, the scrobbler logs and the usual filesystem
metadata directories. The mirror does not contain them, so a sync to the card
root would otherwise have deleted the Rockbox installation -- which the first
draft of this script would have done.
The unmount is why this is a script at all. FAT32 has no journal, the device is
reached through the Apple firmware's disk mode because Rockbox's own mass
storage is unreliable on an iFlash, and an interrupted write is corruption that
needs fsck.vfat from another machine.
tools/submit_scrobbles.py sends the Rockbox scrobbler log to Last.fm and sets
it aside. Rockbox writes it in AUDIOSCROBBLER 1.1: tab-separated, one line per
track, rated L for listened or S for skipped, and only the listened ones are a
play. It runs before the copy, because the plays already happened and a failed
transfer is no reason to lose them as well.
Two things there differ from every other Last.fm call in these projects.
Scrobbling is a write method, so it needs the API secret and a session key
obtained once through the browser rather than the read-only key. And a target
with no real-time clock gets /.scrobbler-timeless.log with every timestamp set
to zero; those are counted and reported but never sent, since submitting them
would mean inventing when they happened.
Signature generation sorts parameter names by the ASCII table rather than
numerically, so artist[10] precedes artist[1]. Sorting them the obvious way
produces an invalid signature and no other symptom, so there is a test for it.
The log is renamed rather than deleted once accepted, so that if Last.fm
quietly dropped something the evidence is still on the device.
Emma Thorpe