Emma Thorpe
37b841f009
Build and publish container / build (pull_request) Successful in 2m33s
The transfer looked hung. rsync prints nothing while it builds its file list, which on fifty thousand files over USB is several minutes of silence, and --info=progress2 does not help: with incremental recursion its percentage is computed against a list rsync has not finished discovering, so it moves backwards as often as forwards. The script now counts what needs copying first and says so, then renders its own single line that rewrites in place, showing the album currently going across and a percentage against a total that is actually known. Counting costs a second pass over the tree. That is the price of a percentage meaning something, and it is cheaper than staring at a blank terminal wondering whether the thing has died. Directories are excluded from the count. rsync reports those too, and including them puts the figure past a hundred per cent. Piped to a log the line becomes a plain one every thirty seconds, because a log full of carriage returns and escape codes is not a log anybody reads.
200 lines
8.0 KiB
Bash
Executable File
200 lines
8.0 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Copy the mirror onto a Rockbox device, then unmount it cleanly.
|
|
#
|
|
# The device is FAT32 with no journal, reached through the Apple firmware's
|
|
# disk mode because Rockbox's own mass storage is unreliable on an iFlash. An
|
|
# interrupted write is corruption that needs fsck.vfat from another machine, so
|
|
# this syncs and unmounts rather than leaving that to whoever pulls the cable.
|
|
#
|
|
# rsync --delete is pointed at a whole filesystem, so the checks below are the
|
|
# point of the script rather than decoration.
|
|
|
|
set -euo pipefail
|
|
|
|
usage() {
|
|
# Help goes to stdout and exits clean; misuse goes to stderr and does not.
|
|
local stream=2 code=2
|
|
if [ "${1:-}" = "help" ]; then
|
|
stream=1
|
|
code=0
|
|
fi
|
|
cat >&"$stream" <<'USAGE'
|
|
usage: sync-to-ipod.sh [options] <mirror> <destination>
|
|
|
|
-n dry run; show what would change and touch nothing
|
|
-f copy even if the FAT32 check finds unacceptable paths
|
|
-S skip submitting the Rockbox scrobbler log to Last.fm
|
|
-U leave the destination mounted afterwards
|
|
|
|
Submitting scrobbles needs LASTFM_API_KEY and LASTFM_API_SECRET; it is skipped
|
|
with a note when they are unset. Scrobbling is a write method and needs the
|
|
secret, unlike the read-only calls elsewhere in these projects.
|
|
|
|
The mirror is the directory holding the artist folders. The destination is
|
|
where those folders should end up on the device -- not the card root, unless
|
|
that is genuinely where you want them:
|
|
|
|
sync-to-ipod.sh /mnt/tank/media/music-mp3 /media/IPOD/Music
|
|
|
|
A subdirectory is the better target: --delete is confined to it, and the path
|
|
budget is derived from it, since the device's 260-character limit counts the
|
|
whole path as the device sees it. /.rockbox and the scrobbler logs are never
|
|
deleted wherever you point this.
|
|
|
|
The destination must be on a mounted FAT filesystem. That check is also what
|
|
catches an unmounted device: /media/IPOD/Music then resolves to the host's own
|
|
root filesystem, and this refuses to empty that.
|
|
|
|
Progress is one line that rewrites itself, showing the album currently going
|
|
across and how far through the transfer is. Working the total out first means a
|
|
second pass over the tree, which is the price of a percentage that means
|
|
something; rsync's own is computed against a file list it is still building.
|
|
|
|
Reach the device with the Apple firmware's disk mode: Menu+Select to reboot,
|
|
then immediately Select+Play. Power off afterwards by holding Play.
|
|
USAGE
|
|
exit "$code"
|
|
}
|
|
|
|
dry_run=false
|
|
force=false
|
|
unmount=true
|
|
scrobble=true
|
|
for argument in "$@"; do
|
|
[ "$argument" = "--help" ] && usage help
|
|
done
|
|
while getopts ":nfSUh" option; do
|
|
case "$option" in
|
|
n) dry_run=true ;;
|
|
f) force=true ;;
|
|
S) scrobble=false ;;
|
|
U) unmount=false ;;
|
|
h) usage help ;;
|
|
*) usage ;;
|
|
esac
|
|
done
|
|
shift $((OPTIND - 1))
|
|
[ $# -eq 2 ] || usage
|
|
|
|
# Trailing slashes are stripped for tidiness, but stripping one from "/" leaves
|
|
# an empty string, and the guard below would then never see the root it is
|
|
# there to refuse.
|
|
mirror=${1%/}
|
|
mirror=${mirror:-/}
|
|
destination=${2%/}
|
|
destination=${destination:-/}
|
|
here=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)
|
|
|
|
die() {
|
|
# Every argument, not just the first: the second half of a message is
|
|
# usually the half that says what to do about it.
|
|
printf 'sync-to-ipod: %s\n' "$*" >&2
|
|
exit 1
|
|
}
|
|
|
|
[ -d "$mirror" ] || die "mirror $mirror is not a directory"
|
|
[ -n "$(ls -A "$mirror")" ] || die "mirror $mirror is empty; refusing to mirror nothing"
|
|
[ -d "$destination" ] || die "destination $destination is not a directory"
|
|
|
|
# --delete makes every one of these load-bearing. Emptying the wrong directory
|
|
# is not a mistake that announces itself.
|
|
case "$destination" in
|
|
"" | "/" | "$HOME") die "refusing to sync onto $destination" ;;
|
|
esac
|
|
[ "$(readlink -f "$mirror")" != "$(readlink -f "$destination")" ] ||
|
|
die "mirror and destination are the same directory"
|
|
|
|
# The filesystem the destination sits on, which is the check that matters: a
|
|
# subdirectory of the card is a perfectly good target, and is the better one,
|
|
# because --delete is then confined to it. Being FAT is also what proves the
|
|
# card is mounted at all -- an unmounted /media/IPOD/Music resolves to the
|
|
# host's own root filesystem, and this refuses to empty that.
|
|
filesystem=$(findmnt -no FSTYPE --target "$destination")
|
|
mounted_on=$(findmnt -no TARGET --target "$destination")
|
|
case "$filesystem" in
|
|
vfat | exfat) ;;
|
|
*)
|
|
$force ||
|
|
die "$destination is on a $filesystem filesystem, not FAT." \
|
|
"Is the device mounted? Pass -f if this is deliberate."
|
|
printf 'sync-to-ipod: destination is %s, not FAT\n' "$filesystem" >&2
|
|
;;
|
|
esac
|
|
|
|
# What the device will call this directory, which is what its path limit
|
|
# applies to. Derived rather than configured, so it cannot disagree with where
|
|
# the files are actually going.
|
|
device_prefix=${destination#"$mounted_on"}
|
|
device_prefix="/${device_prefix#/}"
|
|
printf 'sync-to-ipod: the device will see this as %s\n' "$device_prefix" >&2
|
|
|
|
if $force; then
|
|
printf 'sync-to-ipod: skipping the FAT32 check\n' >&2
|
|
elif ! python3 "$here/check_fat32.py" --device-prefix "$device_prefix" "$mirror"; then
|
|
die "the mirror holds paths FAT32 will not take; run music-mirror with --fat32-safe"
|
|
fi
|
|
|
|
# Before the copy, not after: the plays already happened, and if the transfer
|
|
# then fails there is no reason to have lost them too.
|
|
if $scrobble; then
|
|
if [ -z "${LASTFM_API_KEY:-}" ] || [ -z "${LASTFM_API_SECRET:-}" ]; then
|
|
printf 'sync-to-ipod: no Last.fm credentials, skipping the scrobbler log\n' >&2
|
|
else
|
|
scrobble_options=()
|
|
$dry_run && scrobble_options+=(--dry-run)
|
|
python3 "$here/submit_scrobbles.py" "${scrobble_options[@]}" "$destination" ||
|
|
die "submitting scrobbles failed; nothing has been copied"
|
|
fi
|
|
fi
|
|
|
|
# -rt rather than -a: owners, groups and permissions mean nothing on FAT, and
|
|
# asking for them produces a screenful of errors and a non-zero exit.
|
|
# --modify-window=2 because FAT stores mtimes to two-second resolution, without
|
|
# which every file looks changed and the whole library is copied every time.
|
|
options=(--recursive --times --delete --modify-window=2)
|
|
# --delete removes tracks whose source has gone, which is the point. It would
|
|
# also remove everything on the device that the mirror does not contain -- and
|
|
# if the destination is the card root that means /.rockbox, the Rockbox install
|
|
# itself. Excluded paths are not deleted unless --delete-excluded is given,
|
|
# which it never is here.
|
|
for owned in "/.rockbox" "/.scrobbler.log" "/.scrobbler.log.*" "/.playlist_control" \
|
|
"/System Volume Information" "/.Spotlight-V100" "/.Trashes" "/.fseventsd"; do
|
|
options+=(--exclude "$owned")
|
|
done
|
|
|
|
printf 'sync-to-ipod: %s -> %s\n' "$mirror" "$destination" >&2
|
|
|
|
if $dry_run; then
|
|
rsync "${options[@]}" --dry-run --verbose "$mirror/" "$destination/"
|
|
printf 'sync-to-ipod: dry run, nothing was written\n' >&2
|
|
exit 0
|
|
fi
|
|
|
|
# rsync says nothing at all while it builds its file list, which on fifty
|
|
# thousand files over USB is minutes of apparent hang. Counting first costs a
|
|
# second pass over the tree but means the transfer can show a real percentage
|
|
# rather than a number that grows as rsync discovers more work.
|
|
printf 'sync-to-ipod: working out what needs copying...\n' >&2
|
|
total=$(rsync "${options[@]}" --dry-run --out-format='%n' "$mirror/" "$destination/" |
|
|
grep -cve '/$' || true)
|
|
printf 'sync-to-ipod: %s files to copy\n' "$total" >&2
|
|
|
|
rsync "${options[@]}" --out-format='%n' "$mirror/" "$destination/" |
|
|
python3 "$here/rsync_progress.py" --total "$total"
|
|
status=${PIPESTATUS[0]}
|
|
[ "$status" -eq 0 ] || die "rsync exited $status"
|
|
|
|
sync
|
|
if $unmount; then
|
|
device=$(findmnt -no SOURCE --target "$destination")
|
|
printf 'sync-to-ipod: unmounting %s\n' "$device" >&2
|
|
if command -v udisksctl >/dev/null 2>&1; then
|
|
udisksctl unmount -b "$device"
|
|
else
|
|
umount -- "$destination"
|
|
fi
|
|
printf 'sync-to-ipod: safe to disconnect\n' >&2
|
|
else
|
|
printf 'sync-to-ipod: still mounted; unmount before disconnecting\n' >&2
|
|
fi
|