feat: show which album is copying, and how far through

The transfer looked hung. rsync prints nothing while it builds its file list,
which on fifty thousand files over USB is several minutes of silence, and
--info=progress2 does not help: with incremental recursion its percentage is
computed against a list rsync has not finished discovering, so it moves
backwards as often as forwards.

The script now counts what needs copying first and says so, then renders its
own single line that rewrites in place, showing the album currently going
across and a percentage against a total that is actually known. Counting costs
a second pass over the tree. That is the price of a percentage meaning
something, and it is cheaper than staring at a blank terminal wondering whether
the thing has died.

Directories are excluded from the count. rsync reports those too, and including
them puts the figure past a hundred per cent.

Piped to a log the line becomes a plain one every thirty seconds, because a log
full of carriage returns and escape codes is not a log anybody reads.

tools/rsync_progress.py

@@ -0,0 +1,89 @@
+#!/usr/bin/env python3
+"""Render rsync's per-file output as a single updating status line.
+
+Fed the paths rsync reports with --out-format='%n', one per line. Prints one
+line that rewrites itself, showing how far through the transfer is and which
+album is currently going across, rather than either scrolling fifty thousand
+filenames past or -- as rsync does while it builds its file list -- saying
+nothing at all for several minutes.
+
+Falls back to periodic plain lines when stderr is not a terminal, so a log does
+not fill up with carriage returns.
+"""
+
+import argparse
+import os
+import shutil
+import sys
+import time
+
+
+def album_of(path):
+    """Return "Artist / Album" for a mirror-relative path."""
+    parts = [part for part in path.strip("/").split("/") if part]
+    if len(parts) >= 3:
+        return f"{parts[0]} / {parts[1]}"
+    if len(parts) == 2:
+        return parts[0]
+    return ""
+
+
+def fit(text, width):
+    """Trim to the terminal, from the left: the album matters more than the artist."""
+    if width <= 1 or len(text) <= width:
+        return text
+    return "…" + text[-(width - 1) :]
+
+
+def render(done, total, label, width):
+    if total > 0:
+        share = min(100, done * 100 // total)
+        head = f"[{done:>6,} / {total:<6,} {share:>3}%] "
+    else:
+        head = f"[{done:>6,} files] "
+    return head + fit(label, max(0, width - len(head)))
+
+
+def main(argv=None, stream=None, out=None):
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--total", type=int, default=0, help="files expected")
+    parser.add_argument("--interval", type=float, default=0.1, help="seconds between redraws")
+    args = parser.parse_args(argv)
+
+    stream = stream or sys.stdin
+    out = out or sys.stderr
+    interactive = out.isatty()
+    width = shutil.get_terminal_size((100, 24)).columns - 1
+
+    done = 0
+    last_drawn = 0.0
+    label = ""
+    for line in stream:
+        path = line.rstrip("\n")
+        # rsync reports directories too, with a trailing slash. They are not
+        # files and counting them would put the percentage past a hundred.
+        if not path or path.endswith("/"):
+            continue
+        done += 1
+        label = album_of(path) or os.path.basename(path)
+
+        now = time.monotonic()
+        if interactive:
+            if now - last_drawn >= args.interval:
+                out.write("\r\033[2K" + render(done, args.total, label, width))
+                out.flush()
+                last_drawn = now
+        elif now - last_drawn >= 30:
+            out.write(render(done, args.total, label, width) + "\n")
+            out.flush()
+            last_drawn = now
+
+    if interactive:
+        out.write("\r\033[2K")
+    out.write(render(done, args.total, label, width).rstrip() + "\n")
+    out.flush()
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

tools/sync-to-ipod.sh

@@ -45,6 +45,11 @@ The destination must be on a mounted FAT filesystem. That check is also what
 catches an unmounted device: /media/IPOD/Music then resolves to the host's own
 root filesystem, and this refuses to empty that.
 
+Progress is one line that rewrites itself, showing the album currently going
+across and how far through the transfer is. Working the total out first means a
+second pass over the tree, which is the price of a percentage that means
+something; rsync's own is computed against a file list it is still building.
+
 Reach the device with the Apple firmware's disk mode: Menu+Select to reboot,
 then immediately Select+Play. Power off afterwards by holding Play.
 USAGE
@@ -146,26 +151,39 @@ fi
 # asking for them produces a screenful of errors and a non-zero exit.
 # --modify-window=2 because FAT stores mtimes to two-second resolution, without
 # which every file looks changed and the whole library is copied every time.
+options=(--recursive --times --delete --modify-window=2)
 # --delete removes tracks whose source has gone, which is the point. It would
 # also remove everything on the device that the mirror does not contain -- and
 # if the destination is the card root that means /.rockbox, the Rockbox install
 # itself. Excluded paths are not deleted unless --delete-excluded is given,
 # which it never is here.
-options=(--recursive --times --delete --modify-window=2 --human-readable --info=progress2)
 for owned in "/.rockbox" "/.scrobbler.log" "/.scrobbler.log.*" "/.playlist_control" \
     "/System Volume Information" "/.Spotlight-V100" "/.Trashes" "/.fseventsd"; do
     options+=(--exclude "$owned")
 done
-$dry_run && options+=(--dry-run --verbose)
 
 printf 'sync-to-ipod: %s -> %s\n' "$mirror" "$destination" >&2
-rsync "${options[@]}" "$mirror/" "$destination/"
 
 if $dry_run; then
+    rsync "${options[@]}" --dry-run --verbose "$mirror/" "$destination/"
     printf 'sync-to-ipod: dry run, nothing was written\n' >&2
     exit 0
 fi
 
+# rsync says nothing at all while it builds its file list, which on fifty
+# thousand files over USB is minutes of apparent hang. Counting first costs a
+# second pass over the tree but means the transfer can show a real percentage
+# rather than a number that grows as rsync discovers more work.
+printf 'sync-to-ipod: working out what needs copying...\n' >&2
+total=$(rsync "${options[@]}" --dry-run --out-format='%n' "$mirror/" "$destination/" |
+    grep -cve '/$' || true)
+printf 'sync-to-ipod: %s files to copy\n' "$total" >&2
+
+rsync "${options[@]}" --out-format='%n' "$mirror/" "$destination/" |
+    python3 "$here/rsync_progress.py" --total "$total"
+status=${PIPESTATUS[0]}
+[ "$status" -eq 0 ] || die "rsync exited $status"
+
 sync
 if $unmount; then
     device=$(findmnt -no SOURCE --target "$destination")