fix: copy albums whole when a re-level changes only their tags

A ReplayGain album gain belongs to the whole record, so a track arriving or
leaving rewrites the tags on every one of its siblings. rsgain fits the new
values into the padding its previous write left behind, which changes neither
the file's size nor its mtime:

  before: 277757 bytes, mtime 1577880000, album gain 3.75 dB
  after:  277757 bytes, mtime 1577880000, album gain 6.25 dB

Those are the two things rsync's quick check compares, so the siblings are
invisible to it and the device keeps the old gains indefinitely.

The track that arrived or left is always visible. So sync-to-ipod.sh now runs a
second pass over the albums the first one touched, with --ignore-times to
defeat the same quick check. touched_albums.py derives the list from rsync's
own report of what it moved, which costs no extra traversal of either tree, and
leaves out the tracks the first pass has already copied so a whole-album
quality upgrade is not sent twice. Albums whose file set has not changed are
left alone.

A dry run reports how many further tracks are involved.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

tools/sync-to-ipod.sh

@@ -56,6 +56,10 @@ The destination must be on a mounted FAT filesystem. That check is also what
 catches an unmounted device: /media/IPOD/Music then resolves to the host's own
 root filesystem, and this refuses to empty that.
 
+An album that gained or lost a track is copied again in full afterwards. Its
+surviving tracks have had their ReplayGain tags rewritten in place, which
+changes neither their size nor their mtime, so the main pass cannot see them.
+
 Progress is one line that rewrites itself, showing the album currently going
 across, how far through the transfer is, the rate, and an estimate of what is
 left. Working the totals out first means a second pass over the tree, which is
@@ -193,8 +197,23 @@ done
 
 printf 'sync-to-ipod: %s -> %s\n' "$mirror" "$destination" >&2
 
+# Both passes below feed their file list through touched_albums.py, which reads
+# rsync's own report of what it moved. --files-from separates paths by newline,
+# so a filename containing one would be read as two -- which is a path FAT32
+# will not take either, and check_fat32.py above has already refused the run
+# unless -f was given to skip it.
+changed=$(mktemp)
+relevelled=$(mktemp)
+trap 'rm -f "$changed" "$relevelled"' EXIT
+
 if $dry_run; then
-    rsync "${options[@]}" --dry-run --verbose "$mirror/" "$destination/"
+    rsync "${options[@]}" --dry-run --verbose --out-format='%l %n' \
+        "$mirror/" "$destination/" | tee "$changed"
+    also=$(python3 "$here/touched_albums.py" --mirror "$mirror" <"$changed" | wc -l)
+    if [ "$also" -gt 0 ]; then
+        printf 'sync-to-ipod: and %s more in those albums, whose ReplayGain tags\n' "$also" >&2
+        printf 'sync-to-ipod: change without changing their size or their mtime\n' >&2
+    fi
     printf 'sync-to-ipod: dry run, nothing was written\n' >&2
     exit 0
 fi
@@ -256,10 +275,33 @@ interrupted() {
 trap interrupted INT TERM
 
 rsync "${options[@]}" --out-format='%l %n' "$mirror/" "$destination/" |
+    tee "$changed" |
     python3 "$here/rsync_progress.py" --total "$total" --bytes "$total_bytes"
 status=${PIPESTATUS[0]}
 [ "$status" -eq 0 ] || die "rsync exited $status"
 
+# A ReplayGain album gain belongs to the whole record, so a track arriving or
+# leaving rewrites the tags on all of its siblings. rsgain fits the new values
+# into the padding its previous write left behind, which changes neither the
+# size nor the mtime -- the only two things the pass above compares. Those
+# tracks are invisible to it, and the device would keep the old gains.
+#
+# The track that arrived or left is visible, though. So every album the pass
+# touched has the rest of its tracks copied again, with --ignore-times to
+# defeat the same quick check. No --delete: the pass above has already settled
+# what should be on the device, and --delete aimed at an explicit file list
+# does not mean what it looks like it means.
+python3 "$here/touched_albums.py" --mirror "$mirror" <"$changed" >"$relevelled"
+if [ -s "$relevelled" ]; then
+    also=$(wc -l <"$relevelled")
+    printf 'sync-to-ipod: re-copying %s tracks whose album was re-levelled\n' "$also" >&2
+    rsync --times --modify-window=2 --whole-file --omit-dir-times --ignore-times \
+        --files-from="$relevelled" --out-format='%l %n' "$mirror/" "$destination/" |
+        python3 "$here/rsync_progress.py" --total "$also"
+    status=${PIPESTATUS[0]}
+    [ "$status" -eq 0 ] || die "the re-levelled tracks failed to copy: rsync exited $status"
+fi
+
 # Rockbox reads its database from .tcd files in .rockbox. Building them here
 # rather than on the device is not just faster: the on-device commit sorts the
 # whole index in whatever memory it can scrape together, and on a large library

tools/touched_albums.py

@@ -0,0 +1,97 @@
+#!/usr/bin/env python3
+"""List the tracks a sync has to copy again because their album was re-levelled.
+
+Reads rsync's `--out-format='%l %n'` output on stdin and writes mirror-relative
+paths on stdout, one per line, for feeding straight back to rsync as
+`--files-from`.
+
+The problem it solves: a ReplayGain album gain is a property of every track on
+the record, so one track arriving or leaving changes the tags on all of its
+siblings. rsgain writes the new values into the padding its previous write left
+behind, which leaves both the file's size and its mtime untouched -- and size
+and mtime are exactly what rsync's quick check compares. It sees nothing to do,
+and the device keeps the old gains.
+
+What rsync always can see is the track that arrived or left. So any album it
+touched has the rest of its tracks copied again, and nothing else does.
+
+Tracks rsync has already dealt with are left out of the list. After a quality
+upgrade that replaces every track on a record, listing them again would send
+the whole album twice.
+"""
+
+import argparse
+import re
+import sys
+from pathlib import Path
+
+# Under --out-format='%l %n' a transfer is reported as the size, a space and
+# the path. A removal is reported as "deleting <path>" whatever the format is.
+# Everything else on the stream is rsync talking to the operator -- the file
+# list preamble, the byte totals -- and is not a path.
+TRANSFER = re.compile(r"^(\d+) (.+)$")
+DELETION = re.compile(r"^deleting (.+)$")
+
+# What the mirror is made of, and so the only thing worth copying again. A
+# cover is not rewritten by a re-level.
+MIRROR_SUFFIX = ".mp3"
+
+
+def touched(lines):
+    """Return the paths rsync transferred and the directories it changed."""
+    transferred = set()
+    directories = set()
+
+    for line in lines:
+        line = line.rstrip("\n")
+        deletion = DELETION.match(line)
+        transfer = None if deletion else TRANSFER.match(line)
+        if deletion:
+            path = deletion.group(1)
+        elif transfer:
+            path = transfer.group(2)
+        else:
+            continue
+        # Only a track can change an album's gains. rsync reports the
+        # directories it creates and removes too, and a cover replaced on its
+        # own is no reason to send the record again.
+        if not path.endswith(MIRROR_SUFFIX):
+            continue
+        if transfer:
+            transferred.add(path)
+        directories.add(path.rpartition("/")[0])
+
+    return transferred, directories
+
+
+def remaining(mirror, transferred, directories):
+    """Return the tracks in those directories that rsync has not just copied."""
+    paths = []
+    for directory in sorted(directories):
+        album = mirror / directory if directory else mirror
+        if not album.is_dir():
+            # Removed along with the last of its tracks. Nothing to copy.
+            continue
+        for track in sorted(album.glob(f"*{MIRROR_SUFFIX}")):
+            relative = f"{directory}/{track.name}" if directory else track.name
+            if relative not in transferred:
+                paths.append(relative)
+    return paths
+
+
+def main(argv=None, stream=None, out=None):
+    parser = argparse.ArgumentParser(
+        prog="touched_albums.py",
+        description="List the tracks to copy again after an album was re-levelled.",
+    )
+    parser.add_argument("--mirror", required=True, type=Path, help="root of the MP3 mirror")
+    args = parser.parse_args(argv)
+
+    transferred, directories = touched(stream if stream is not None else sys.stdin)
+    for path in remaining(args.mirror, transferred, directories):
+        print(path, file=out if out is not None else sys.stdout)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())