fix: correct Rockbox's local wall-clock timestamps to UTC

Rockbox has no concept of a timezone. Its clock holds local time, and it builds log timestamps with mktime(get_time()) -- but firmware/libc/mktime.c is plain calendar arithmetic applying no offset, so the RTC's local fields come out as though they were UTC. The number in the log is ahead of the real instant by whatever the offset was, and Last.fm stores UTC, so every play submitted during BST landed an hour in the future. Rockbox states this itself: its scrobbler plugin writes #TZ/UNKNOWN, and the AUDIOSCROBBLER spec allows #TZ/UTC only for a device that actually converted. The correction belongs to the consumer. Each timestamp is decoded back to its wall-clock fields and reinterpreted in the player's zone, per play rather than as one offset over the whole log, so a log spanning a daylight saving change converts each side correctly. A log declaring #TZ/UTC is left alone rather than shifted twice. The zone defaults to this machine's, overridable with --device-timezone or ROCKBOX_TIMEZONE. Deriving it needs the whole IANA name: /etc/localtime resolves into the tzdata tree, and taking only the final component yields "London", which no database holds, silently falling back to a fixed offset that is wrong for half the year. Since Rockbox cannot adjust for daylight saving on its own, the player's clock has to be changed by hand twice a year. Any play converting to a future time is now reported, which is what a forgotten adjustment looks like. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 14:18:52 +01:00
13 changed files with 349 additions and 868 deletions
@@ -7,10 +7,8 @@ FROM python:3.13-alpine AS runtime
 ENV PYTHONUNBUFFERED=1
-# ffmpeg does the encoding and rsgain the volume levelling; the application
+# ffmpeg does the encoding; the application itself has no Python dependencies.
-# itself has no Python dependencies. rsgain lives in the community repository,
+RUN apk add --no-cache ffmpeg
 # which the official Python images already have enabled.
 RUN apk add --no-cache ffmpeg rsgain
 WORKDIR /app
 COPY pyproject.toml README.md ./
@@ -22,7 +22,6 @@ remembering to do anything.
 | Mirror up to date                | skip                                     |
 | Source is already MP3            | copy verbatim                            |
 | Source gone                      | delete the mirror file, prune empty dirs |
 | An album gained or lost a track  | re-measure its ReplayGain, tags only     |
 Freshness is modification time: an encoded file is stamped with its source's
 mtime, so a file is stale exactly when the two differ. There is no database to
@@ -95,7 +94,6 @@ music-mirror --source /music --mirror /music-mp3 --subdir "Artist/Album"
 | `--interval` | `MUSIC_MIRROR_INTERVAL` | unset     | Repeat forever, e.g. `45m`, `6h`, `1d`         |
 | `--subdir`   | —                       | unset     | Limit the pass to one directory; skips pruning |
 | `--fat32-safe` | `MUSIC_MIRROR_FAT32_SAFE` | off     | Name files so a FAT32 device accepts them      |
 | `--no-replaygain` | `MUSIC_MIRROR_REPLAYGAIN` | on   | Write ReplayGain tags; set the variable to `0` to skip |
 | `--no-prune` | —                       | off       | Keep mirror files whose source has gone        |
 | `--dry-run`  | —                       | off       | Report what would change, write nothing        |
@@ -116,9 +114,7 @@ The first full pass is the expensive one; after that only new and changed files
 are touched. Lower `MUSIC_MIRROR_JOBS` if you would rather the NAS stayed
 responsive than finished sooner.
-Requires `ffmpeg` and `ffprobe` on `PATH`, and `rsgain` for volume levelling.
+Requires `ffmpeg` and `ffprobe` on `PATH`. The container image provides both.
 The container image provides all three. A missing `rsgain` is reported once per
 pass and costs the ReplayGain tags; everything else still runs.
 ## Running it on TrueNAS Scale
@@ -197,16 +193,6 @@ flight; only the two figures that needed the second walk are missing.
 Piped to a log it prints a plain line every thirty seconds instead, with no
 carriage returns, and a summary at the end either way.
 ### Albums that changed are copied whole
 After the main pass, any album that gained or lost a track has the rest of its
 tracks copied again with `--ignore-times`. Their ReplayGain tags were rewritten
 in place when the album was re-levelled, which changes neither their size nor
 their mtime — the only two things rsync compares — so nothing else would ever
 send them. `touched_albums.py` works out the list from rsync's own report of
 what it moved, so this costs no extra traversal, and tracks the main pass has
 already copied are left out of it. See [Volume levelling](#volume-levelling).
 ### The Rockbox database
 Point `MUSIC_MIRROR_DATABASE_TOOL` at Rockbox's host-side builder and the sync
@@ -323,6 +309,47 @@ real-time clock Rockbox writes `/.scrobbler-timeless.log` with every timestamp
 set to zero; those are counted and reported but never submitted, because
 scrobbling them would mean inventing when they happened.
 ### Timestamps are local wall clock, and are corrected here
 Rockbox has no concept of a timezone. Its clock is set to local time, and it
 builds log timestamps with `mktime(get_time())` — but
 [its `mktime`](https://git.rockbox.org/cgit/rockbox.git/tree/firmware/libc/mktime.c)
 is plain calendar arithmetic that applies no offset, so the RTC's local fields
 come out as if they were UTC. The number in the log is therefore ahead of the
 real instant by whatever the offset was. Last.fm stores UTC, so submitting it
 raw puts every play an hour into the future for the half of the year the UK is
 on BST.
 Rockbox is candid about this: its scrobbler plugin writes `#TZ/UNKNOWN` in the
 log header, and the AUDIOSCROBBLER spec says a device may claim `#TZ/UTC` only
 if it actually converted. The correction is the consumer's job.
 Each timestamp is decoded back into the wall-clock fields it came from and
 reinterpreted in the player's zone. Doing it **per play** rather than applying
 one offset to the whole log matters: a week's listening can straddle a daylight
 saving change, and the two sides need different offsets. A log that declares
 `#TZ/UTC` is left alone, so a client that already converted is not shifted
 twice.
 The zone defaults to this machine's. Set `ROCKBOX_TIMEZONE` (or pass
 `--device-timezone`) to an IANA name if the player's clock is keeping a
 different one.
 Because Rockbox cannot adjust for daylight saving itself, **you have to change
 the player's clock by hand twice a year**. If you forget, its times are an hour
 out and no amount of zone arithmetic recovers them. The submitter reports any
 play that converts to a time in the future, which is what a forgotten
 adjustment looks like:
 ```
  37 plays are timestamped up to 58 minutes in the future, converting from
  Europe/London. Either the player's clock is wrong or that is not the zone
  it is set to.
 ```
 `--dry-run` prints each play's local time beside the epoch, so the conversion
 can be checked against when you actually remember listening.
 ### Nothing played is thrown away
 Two separate obligations, because a play that happened and never reached
@@ -361,8 +388,6 @@ docker build --target test .   # what CI runs
 pytest                         # needs ffmpeg and pytest on PATH
 ```
 The ReplayGain tests need `rsgain` as well and skip without it.
 The suite runs real ffmpeg encodes rather than mocking them. The interesting
 failures are in what ffmpeg actually does with tags, cover art and container
 formats, and a mock cannot fail that way — which is also why CI runs the tests
@@ -374,7 +399,7 @@ Run them directly instead if you prefer; they skip when ffmpeg is absent. On a
 Nix machine:
 ```sh
-nix shell nixpkgs#python3Packages.pytest nixpkgs#ffmpeg nixpkgs#rsgain -c pytest
+nix shell nixpkgs#python3Packages.pytest nixpkgs#ffmpeg -c pytest
 ```
 ## FAT32 and Rockbox
@@ -464,64 +489,6 @@ firmware reads the embedded one; Rockbox reads the file. Both are satisfied.
 A cover left behind in a directory whose tracks have all gone is pruned, or the
 directory would never look empty and never be removed.
 ### Volume levelling
 Rockbox can level the volume between tracks, but only from tags. It applies the
 offset a ReplayGain tag carries and has no loudness analysis of its own, so a
 mirror without those tags plays every album at whatever level it was mastered
 to — and a 2008 remaster next to a 1972 pressing is a reach for the volume
 wheel on every track change.
 The tags are therefore written here, with `rsgain`, once an album's tracks are
 in place. Both album gain and track gain are measured: album gain preserves the
 quiet track that a record is supposed to have, track gain is the one that makes
 sense on shuffle, and which of them is used is the device's decision, not this
 one.
 Turn it on at the player end under **Settings → Playback Settings →
 Replaygain**:
 | Setting          | Suggested                    | Why                                                        |
 | ---------------- | ---------------------------- | ---------------------------------------------------------- |
 | Replaygain type  | `Track Gain if Shuffling`    | Album gain while playing a record, track gain once shuffle is on — the only setting that is right in both cases |
 | Prevent clipping | `Yes`                        | Backs the gain off using the peak tags rather than distorting |
 | Pre-amp          | `0 dB`                       | The reference is already −18 LUFS; raise it only if everything ends up too quiet |
 `Track Gain if Shuffling` is not a compromise between the other two — Rockbox
 reads the shuffle setting and picks whole-hog album or track gain from it
 (`apps/misc.c`, `replaygain_setting_mode`). It is also Rockbox's default, so on
 a fresh install there may be nothing to change but `Prevent clipping`.
 Measuring costs a full decode of every track, so the first pass after enabling
 it takes roughly as long as the original encode did. After that only albums
 that gained, lost or replaced a track are re-measured. An album is re-measured
 as a whole, because album gain is a property of all of its tracks and one new
 track makes the value stored on every sibling wrong.
 Tagging rewrites the file, and staleness here is an mtime comparison, so
 `rsgain` is run with `--preserve-mtimes`. Without it every levelled track would
 look newer than its source and the next pass would re-encode the entire
 library.
 That has a consequence for the sync, and it is not obvious. The first time a
 track is levelled its tag grows by about a kilobyte, so its size changes and
 rsync copies it — the whole library goes across once, and there is no way
 around that; the tag sits at the head of the file and every byte after it
 moves. But `rsgain` leaves padding behind, so a *later* re-level fits inside it
 and changes neither the size nor the mtime:
 ```
 before re-level: 277757 bytes, mtime 1577880000, album gain 3.75 dB
 after  re-level: 277757 bytes, mtime 1577880000, album gain 6.25 dB
 ```
 Those are the two things rsync's quick check compares, so it would see nothing
 to do and the device would keep the old gains. `sync-to-ipod.sh` handles it: the
 track that arrived or left is always visible, so any album the main pass
 touched has the rest of its tracks copied again with `--ignore-times`. Albums
 whose file set has not changed are not touched, which is what keeps this from
 being a full re-copy.
 ## Getting the result onto an iPod
 The mirror is just a directory of MP3s, so any client will do:
@@ -561,6 +528,6 @@ The mirror is just a directory of MP3s, so any client will do:
 Neither client transcodes at sync time; they copy finished MP3s.
 Two device-side details worth knowing: the iPod reads cover art from the file's
-tags and ignores `folder.jpg`, which is why art is embedded here; and the Apple
+tags and ignores `folder.jpg`, which is why art is embedded here; and volume
-firmware levels volume from iTunes' Soundcheck tag rather than ReplayGain, so
+levelling on the device uses iTunes' Soundcheck tag, not ReplayGain, so
-the tags written here do nothing until the iPod is running Rockbox.
+ReplayGain tags in the source are not carried over as such.
@@ -31,11 +31,6 @@ services:
      # enabling it is quick -- but it is a one-way change to every such path,
      # so decide before running it rather than after.
      MUSIC_MIRROR_FAT32_SAFE: "true"
      # ReplayGain tags, so Rockbox can level the volume between albums. On by
      # default; set to 0 to skip the measuring pass. The first pass after
      # enabling it levels the whole library, which costs a decode of every
      # track, so expect it to take about as long as the original encode.
      # MUSIC_MIRROR_REPLAYGAIN: "0"
    volumes:
      - /mnt/tank/media/music:/music:ro
      - /mnt/tank/media/music-mp3:/mirror
@@ -11,11 +11,6 @@ writes to the source library.
 Staleness is tracked by modification time: an encoded file is given its
 source's mtime, so a file is out of date exactly when the two differ. That
 makes runs idempotent without a database to keep in step.
 Finished albums are levelled with rsgain, which writes ReplayGain tags into the
 mirror. Rockbox applies the offset those tags carry but has no loudness
 analysis of its own, so without them every album plays at whatever level it was
 mastered to.
 """
 import argparse
@@ -86,21 +81,6 @@ MIRROR_COVER = "cover.jpg"
 # Files the mirror is allowed to contain, and therefore allowed to delete.
 MIRROR_SUFFIX = ".mp3"
 # Measures loudness and writes the ReplayGain tags. Not a hard requirement: a
 # pass without it still produces a correct mirror, only one the player cannot
 # level, so a missing binary is a warning rather than a failure.
 REPLAYGAIN_TOOL = "rsgain"
 # Looked for in a file's ID3v2 tag to tell a levelled track from an unlevelled
 # one. Album gain rather than track gain because the album value is the one
 # this writes for; a file carrying only track gain came from somewhere else and
 # should be rescanned.
 REPLAYGAIN_TAG = b"replaygain_album_gain"
 # Enough of a TXXX frame body to hold the encoding byte and the description.
 # The value after it says what the gain is, which is not the question here.
 TXXX_DESCRIPTION_BYTES = 128
 # Filesystems disagree about mtime precision; SMB in particular rounds.
 MTIME_TOLERANCE_SECONDS = 2
@@ -561,13 +541,8 @@ def prune(mirror_root, expected, dry_run):
    Driven by the set of paths the pass expects to exist rather than by
    probing the source tree for names, which would disagree with it over
    letter case and over any extension the walker does not collect.
    Returns the number of files removed and the directories they came out of.
    Losing a track changes an album's loudness, so those directories need
    levelling again even though nothing was written into them.
    """
    removed = 0
    emptied = set()
    for mirror in sorted(mirror_root.rglob(f"*{MIRROR_SUFFIX}")):
        if mirror in expected:
@@ -578,7 +553,6 @@ def prune(mirror_root, expected, dry_run):
            continue
        logger.info("removing orphan %s", mirror)
        mirror.unlink(missing_ok=True)
        emptied.add(mirror.parent)
    if not dry_run:
        # A cover copied for an album whose tracks have all gone is an orphan
@@ -593,164 +567,7 @@ def prune(mirror_root, expected, dry_run):
            if directory.is_dir() and not any(directory.iterdir()):
                directory.rmdir()
-    return removed, emptied
+    return removed
 def syncsafe(data):
    """Return the integer held in syncsafe bytes: seven bits of each."""
    value = 0
    for byte in data:
        value = (value << 7) | (byte & 0x7F)
    return value
 def has_replaygain(path):
    """Return whether an MP3 already carries ReplayGain tags.
    Walks the ID3v2 frame headers and seeks over the bodies rather than reading
    the tag whole. Every file in this mirror has its cover art embedded, so the
    tag is routinely half a megabyte; reading all of it for every track on
    every pass would turn an idle pass into a full read of the library.
    """
    try:
        with open(path, "rb") as handle:
            header = handle.read(10)
            if len(header) < 10 or header[:3] != b"ID3" or header[3] not in (3, 4):
                return False
            remaining = syncsafe(header[6:10])
            # Unsynchronisation shifts every offset in the tag, and the two
            # versions describe an extended header differently. Nothing that
            # writes this mirror emits either, so reading the tag whole is a
            # cheaper answer than the code to walk one that does.
            if header[5] & 0xC0:
                return REPLAYGAIN_TAG in handle.read(remaining).lower()
            while remaining >= 10:
                frame = handle.read(10)
                remaining -= 10
                # Frame ids are upper-case letters and digits, so anything else
                # is the padding that follows the last frame.
                if len(frame) < 10 or not frame[:4].isalnum():
                    return False
                # 2.3 sizes count all eight bits per byte; 2.4 made them
                # syncsafe like the tag length above.
                length = (
                    int.from_bytes(frame[4:8], "big")
                    if header[3] == 3
                    else syncsafe(frame[4:8])
                )
                if length <= 0 or length > remaining:
                    return False
                if frame[:4] == b"TXXX":
                    body = handle.read(min(length, TXXX_DESCRIPTION_BYTES))
                    handle.seek(length - len(body), os.SEEK_CUR)
                    if REPLAYGAIN_TAG in body.lower():
                        return True
                else:
                    handle.seek(length, os.SEEK_CUR)
                remaining -= length
    except OSError:
        return False
    return False
 def replaygain_albums(expected, written):
    """Return the album directories needing a scan, each with its tracks.
    A directory is scanned when this pass changed what is in it, because album
    gain is a property of the whole album: one track added, replaced or removed
    makes the value stored on every one of its siblings wrong. It is also
    scanned when a track in it has never been levelled, which is what backfills
    a mirror built before any of this existed.
    """
    albums = {}
    for mirror in expected:
        albums.setdefault(mirror.parent, []).append(mirror)
    needed = {}
    for directory, tracks in sorted(albums.items()):
        # A dry run reaches here before anything has been encoded, so the
        # tracks a changed album is going to hold do not exist yet.
        present = sorted(track for track in tracks if track.is_file())
        if directory in written:
            needed[directory] = present
        elif present and not all(map(has_replaygain, present)):
            needed[directory] = present
    return needed
 def replaygain_command(tracks):
    """Return the rsgain command that levels one album directory."""
    return [
        REPLAYGAIN_TOOL,
        "custom",
        # Album mode writes the per-track tags as well as the album ones, so
        # the device is left to choose between them -- Rockbox can apply track
        # gain when shuffling and album gain otherwise, and only if both are
        # present.
        "--album",
        "--tagmode=i",
        # The mirror is ID3v2.3 for the iPod firmware's sake. rsgain would
        # otherwise keep whatever version it found, and "whatever it found" is
        # not a guarantee.
        "--id3v2-version=3",
        # Staleness here is an mtime comparison and tagging rewrites the file.
        # Without this every levelled track would look newer than its source
        # and the next pass would re-encode the entire library, forever.
        "--preserve-mtimes",
        "--quiet",
        *[str(track) for track in tracks],
    ]
 def scan_album(directory, tracks):
    """Write ReplayGain tags across one album. Returns whether it worked."""
    completed = subprocess.run(replaygain_command(tracks), capture_output=True, text=True)
    if completed.returncode != 0:
        lines = completed.stderr.strip().splitlines()
        logger.warning("could not level %s: %s", directory, lines[-1] if lines else "rsgain failed")
        return False
    logger.info("levelled %s", directory)
    return True
 def replaygain(expected, written, jobs, dry_run):
    """Write ReplayGain tags into the albums that need them. Returns how many.
    A failure here is reported and then left alone. The mirror is still correct
    audio in the right place; it just plays at the level it was mastered to,
    which is what every pass before this one produced.
    """
    albums = {
        directory: tracks
        for directory, tracks in replaygain_albums(expected, written).items()
        if tracks or dry_run
    }
    if not albums:
        return 0
    if dry_run:
        logger.info("would level %d album%s", len(albums), "" if len(albums) == 1 else "s")
        return len(albums)
    if shutil.which(REPLAYGAIN_TOOL) is None:
        logger.warning(
            "%s is not on PATH; %d albums are left without ReplayGain tags",
            REPLAYGAIN_TOOL,
            len(albums),
        )
        return 0
    levelled = 0
    with concurrent.futures.ThreadPoolExecutor(max_workers=jobs) as pool:
        futures = [
            pool.submit(scan_album, directory, tracks) for directory, tracks in albums.items()
        ]
        for future in concurrent.futures.as_completed(futures):
            if future.result():
                levelled += 1
    return levelled
 def run_once(
@@ -763,7 +580,6 @@ def run_once(
    do_prune,
    safe=False,
    budget=0,
    do_replaygain=True,
 ):
    """Run a single pass. Returns the number of failures.
@@ -774,7 +590,6 @@ def run_once(
    logger.info("pass starting with %d concurrent encoders", jobs)
    counts = {"encoded": 0, "copied": 0, "renamed": 0, "skipped": 0, "failed": 0}
    failures = []
    written = set()
    work = plan(scan_root, source_root, mirror_root, safe, budget)
@@ -802,8 +617,6 @@ def run_once(
            counts[result.action] += 1
            if result.action == "failed":
                failures.append(result)
            elif result.action != "skipped":
                written.add(result.path.parent)
    expected = set(work)
    if safe and dry_run:
@@ -814,25 +627,20 @@ def run_once(
        for source in work.values():
            expected.add(mirror_path_for(source, source_root, mirror_root))
            expected.add(mirror_path_for(source, source_root, mirror_root, True))
-    removed, emptied = prune(mirror_root, expected, dry_run) if do_prune else (0, set())
+    removed = prune(mirror_root, expected, dry_run) if do_prune else 0
    # After pruning, so an album is not measured with a track in it that is
    # about to be deleted.
    levelled = replaygain(expected, written | emptied, jobs, dry_run) if do_replaygain else 0
    for failure in failures:
        logger.error("failed: %s: %s", failure.path, failure.error)
    logger.info(
        "pass complete in %.1fs: %d encoded, %d copied, %d renamed, %d up to date,"
-        " %d removed, %d levelled, %d failed",
+        " %d removed, %d failed",
        time.monotonic() - started,
        counts["encoded"],
        counts["copied"],
        counts["renamed"],
        counts["skipped"],
        removed,
        levelled,
        counts["failed"],
    )
    return counts["failed"]
@@ -932,13 +740,6 @@ def build_parser():
        action="store_true",
        help="keep mirror files whose source has been deleted",
    )
    parser.add_argument(
        "--no-replaygain",
        action="store_true",
        default=os.getenv("MUSIC_MIRROR_REPLAYGAIN", "").lower() in ("0", "false", "no"),
        help="do not write ReplayGain tags; skips the rsgain pass over changed"
        " albums (env MUSIC_MIRROR_REPLAYGAIN=0)",
    )
    parser.add_argument(
        "--dry-run",
        action="store_true",
@@ -1030,7 +831,6 @@ def main(argv=None):
                do_prune,
                args.fat32_safe,
                budget,
                not args.no_replaygain,
            )
            if interval is None or stopping:
                return 1 if failures else 0
@@ -4,12 +4,12 @@ build-backend = "setuptools.build_meta"
 [project]
 name = "music-mirror"
-version = "0.5.0"
+version = "0.4.0"
 description = "Maintain a lossy MP3 mirror of a lossless music library"
 readme = "README.md"
 requires-python = ">=3.11"
-# No runtime Python dependencies: the work is done by ffmpeg and rsgain, which
+# No runtime Python dependencies: the work is done by ffmpeg, which must be on
-# must be on PATH. A missing rsgain costs the ReplayGain tags and nothing else.
+# PATH.
 dependencies = []
 [project.scripts]
@@ -19,13 +19,6 @@ def require_ffmpeg():
            pytest.skip(f"{tool} is not on PATH", allow_module_level=True)
@pytest.fixture
 def require_rsgain():
    """Skip a test that measures loudness for real rather than faking it."""
    if shutil.which("rsgain") is None:
        pytest.skip("rsgain is not on PATH")
@pytest.fixture
 def tight_umask():
    """Run a test under a umask that would otherwise make the mirror private."""
@@ -53,14 +46,7 @@ def owner_hostile_umask():
 def make_flac():
    """Return a factory writing a short tagged FLAC file."""
-    def factory(
+    def factory(path, title="Test Title", artist="Test Artist", album="Test Album", seconds=1):
        path,
        title="Test Title",
        artist="Test Artist",
        album="Test Album",
        seconds=1,
        gain=0,
    ):
        path.parent.mkdir(parents=True, exist_ok=True)
        subprocess.run(
            [
@@ -74,9 +60,6 @@ def make_flac():
                "lavfi",
                "-i",
                f"sine=frequency=440:duration={seconds}",
                # Quieter or louder than the default, for tests that need two
                # tracks at different levels.
                *(["-af", f"volume={gain}dB"] if gain else []),
                "-metadata",
                f"title={title}",
                "-metadata",
@@ -14,16 +14,6 @@ def run(source, mirror, *extra):
    return music_mirror.main(["--source", str(source), "--mirror", str(mirror), *extra])
 def audio_frames(path):
    """Return a hash of a file's audio frames, ignoring its tags."""
    return subprocess.run(
        ["ffmpeg", "-v", "error", "-i", str(path), "-map", "0:a", "-c", "copy", "-f", "md5", "-"],
        check=True,
        capture_output=True,
        text=True,
    ).stdout.strip()
 def test_parse_quality_accepts_vbr_and_cbr():
    assert music_mirror.parse_quality("V0") == ["-q:a", "0"]
    assert music_mirror.parse_quality("v2") == ["-q:a", "2"]
@@ -71,10 +61,8 @@ def test_output_is_mp3(tmp_path, make_flac):
            "a:0",
            "-show_entries",
            "stream=codec_name",
            # Not csv: a levelled file carries ReplayGain side data, which the
            # csv writer renders as a trailing empty field.
            "-of",
-            "default=noprint_wrappers=1:nokey=1",
+            "csv=p=0",
            str(mirror / "a.mp3"),
        ],
        check=True,
@@ -152,9 +140,6 @@ def test_no_prune_keeps_orphans(tmp_path, make_flac):
 def test_existing_mp3_is_copied_not_re_encoded(tmp_path, make_flac):
    """Compared by the audio frames rather than the whole file: the copy is
    tagged with its ReplayGain values afterwards, so the two differ in the
    container while carrying identical audio."""
    source = tmp_path / "src"
    mirror = tmp_path / "dst"
    flac = make_flac(source / "a.flac")
@@ -167,7 +152,7 @@ def test_existing_mp3_is_copied_not_re_encoded(tmp_path, make_flac):
    run(source, mirror)
-    assert audio_frames(mirror / "b.mp3") == audio_frames(source / "b.mp3")
+    assert (mirror / "b.mp3").read_bytes() == (source / "b.mp3").read_bytes()
 def test_interrupted_copy_leaves_nothing_behind(tmp_path, make_flac, monkeypatch):
@@ -801,181 +786,3 @@ def test_the_budget_is_reported_so_it_can_be_checked(tmp_path, make_flac, caplog
        run(source, mirror, "--fat32-safe")
    assert "limited to 253 characters" in caplog.text
 # Rockbox applies the offset a ReplayGain tag carries but never measures
 # loudness itself, so an untagged mirror plays each album at whatever level it
 # was mastered to. The tags have to be written here or nowhere.
 def test_replaygain_tags_are_written(tmp_path, make_flac, probe_tag, require_rsgain):
    source = tmp_path / "src"
    mirror = tmp_path / "dst"
    make_flac(source / "Album" / "a.flac")
    run(source, mirror)
    track = mirror / "Album" / "a.mp3"
    assert probe_tag(track, "REPLAYGAIN_TRACK_GAIN").endswith("dB")
    assert probe_tag(track, "REPLAYGAIN_ALBUM_GAIN").endswith("dB")
    assert probe_tag(track, "REPLAYGAIN_TRACK_PEAK")
 def test_the_album_shares_one_gain_and_the_tracks_keep_their_own(
    tmp_path, make_flac, probe_tag, require_rsgain
 ):
    """Album gain is what keeps a quiet track quiet within a record it belongs
    to. Track gain is written alongside it so the device can pick the other
    behaviour when shuffling."""
    source = tmp_path / "src"
    mirror = tmp_path / "dst"
    make_flac(source / "Album" / "loud.flac")
    make_flac(source / "Album" / "quiet.flac", gain=-12)
    run(source, mirror)
    loud = mirror / "Album" / "loud.mp3"
    quiet = mirror / "Album" / "quiet.mp3"
    assert probe_tag(loud, "REPLAYGAIN_ALBUM_GAIN") == probe_tag(quiet, "REPLAYGAIN_ALBUM_GAIN")
    assert probe_tag(loud, "REPLAYGAIN_TRACK_GAIN") != probe_tag(quiet, "REPLAYGAIN_TRACK_GAIN")
 def test_levelling_does_not_make_the_next_pass_re_encode(
    tmp_path, make_flac, caplog, require_rsgain
 ):
    """Writing tags rewrites the file, and staleness here is an mtime
    comparison. Without --preserve-mtimes every pass would re-encode the whole
    library and then level it again, forever."""
    source = tmp_path / "src"
    mirror = tmp_path / "dst"
    make_flac(source / "Album" / "a.flac")
    run(source, mirror)
    before = (mirror / "Album" / "a.mp3").stat().st_mtime
    with caplog.at_level("INFO"):
        run(source, mirror)
    assert "0 encoded" in caplog.text
    assert (mirror / "Album" / "a.mp3").stat().st_mtime == before
 def test_an_already_levelled_album_is_not_measured_again(
    tmp_path, make_flac, caplog, require_rsgain
 ):
    """Measuring costs a decode of every track. A pass that changed nothing
    must not pay it."""
    source = tmp_path / "src"
    mirror = tmp_path / "dst"
    make_flac(source / "Album" / "a.flac")
    run(source, mirror)
    with caplog.at_level("INFO"):
        run(source, mirror)
    assert "0 levelled" in caplog.text
 def test_a_new_track_relevels_the_album_around_it(
    tmp_path, make_flac, probe_tag, require_rsgain
 ):
    """Album gain is a property of the whole album, so a track arriving late
    makes the value stored on every one of its siblings wrong.
    -10 dB rather than something more dramatic: R128 gates quiet passages out
    of the measurement, and a track far enough below the rest of the record is
    excluded from it entirely."""
    source = tmp_path / "src"
    mirror = tmp_path / "dst"
    make_flac(source / "Album" / "a.flac")
    run(source, mirror)
    first = probe_tag(mirror / "Album" / "a.mp3", "REPLAYGAIN_ALBUM_GAIN")
    make_flac(source / "Album" / "b.flac", gain=-10)
    run(source, mirror)
    assert probe_tag(mirror / "Album" / "a.mp3", "REPLAYGAIN_ALBUM_GAIN") != first
 def test_a_removed_track_relevels_the_album_behind_it(
    tmp_path, make_flac, caplog, require_rsgain
 ):
    source = tmp_path / "src"
    mirror = tmp_path / "dst"
    make_flac(source / "Album" / "a.flac")
    make_flac(source / "Album" / "b.flac", gain=-20)
    run(source, mirror)
    (source / "Album" / "b.flac").unlink()
    with caplog.at_level("INFO"):
        run(source, mirror)
    assert "1 levelled" in caplog.text
 def test_embedded_cover_art_does_not_hide_the_tags(
    tmp_path, make_flac, make_cover, require_rsgain
 ):
    """The check walks ID3v2 frame headers and seeks over the bodies. Cover art
    sits between the text frames and the ones rsgain appends, so a check that
    only read the start of the tag would never reach them -- and would measure
    every album with a cover on every pass."""
    source = tmp_path / "src"
    mirror = tmp_path / "dst"
    make_flac(source / "Album" / "a.flac")
    make_cover(source / "Album" / "cover.jpg")
    run(source, mirror)
    assert music_mirror.has_replaygain(mirror / "Album" / "a.mp3")
 def test_no_replaygain_leaves_the_tags_off(tmp_path, make_flac, probe_tag):
    source = tmp_path / "src"
    mirror = tmp_path / "dst"
    make_flac(source / "Album" / "a.flac")
    run(source, mirror, "--no-replaygain")
    assert not probe_tag(mirror / "Album" / "a.mp3", "REPLAYGAIN_ALBUM_GAIN")
    assert not music_mirror.has_replaygain(mirror / "Album" / "a.mp3")
 def test_an_unlevelled_mirror_is_backfilled(tmp_path, make_flac, probe_tag, require_rsgain):
    """A mirror built before any of this existed has correct mtimes, so no pass
    would ever revisit those files on its own."""
    source = tmp_path / "src"
    mirror = tmp_path / "dst"
    make_flac(source / "Album" / "a.flac")
    run(source, mirror, "--no-replaygain")
    run(source, mirror)
    assert probe_tag(mirror / "Album" / "a.mp3", "REPLAYGAIN_ALBUM_GAIN").endswith("dB")
 def test_a_missing_scanner_is_reported_and_not_fatal(tmp_path, make_flac, caplog, monkeypatch):
    """The mirror is still correct audio in the right place. It just plays at
    the level it was mastered to."""
    source = tmp_path / "src"
    mirror = tmp_path / "dst"
    make_flac(source / "Album" / "a.flac")
    monkeypatch.setattr(music_mirror.shutil, "which", lambda name: None)
    with caplog.at_level("INFO"):
        assert run(source, mirror) == 0
    assert "rsgain is not on PATH" in caplog.text
    assert (mirror / "Album" / "a.mp3").is_file()
 def test_a_dry_run_measures_nothing(tmp_path, make_flac, caplog, require_rsgain):
    source = tmp_path / "src"
    mirror = tmp_path / "dst"
    make_flac(source / "Album" / "a.flac")
    with caplog.at_level("INFO"):
        run(source, mirror, "--dry-run")
    assert "would level 1 album" in caplog.text
    assert not mirror.joinpath("Album").exists()
@@ -1,7 +1,9 @@
 import json
 import subprocess
 import sys
 from datetime import datetime, timezone
 from pathlib import Path
 from zoneinfo import ZoneInfo
 import pytest
@@ -289,6 +291,130 @@ def test_a_track_missing_from_the_mirror_is_counted_not_guessed(monkeypatch):
    assert len(result.retain) == 1
 LONDON = ZoneInfo("Europe/London")
 # Rockbox builds timestamps with mktime(get_time()), and its mktime applies no
 # zone at all, so the number is the RTC's local wall clock read as if it were
 # UTC. Under BST that puts every play an hour ahead of when it happened.
 SUMMER_LOGGED = 1787839200   # 2026-08-27 14:00 written by the device
 SUMMER_TRUE = 1787835600     # the instant that actually was, 13:00 UTC
 WINTER_LOGGED = 1796479200   # 2026-12-05 14:00, when London is already UTC
 WINTER_TRUE = 1796479200
 def test_a_summer_timestamp_is_pulled_back_to_real_utc():
    """The device logged 14:00 local. That was 13:00 UTC, and UTC is what
    Last.fm stores."""
    assert submit_scrobbles.device_time_to_utc(SUMMER_LOGGED, LONDON) == SUMMER_TRUE
 def test_a_winter_timestamp_is_left_alone():
    """London keeps UTC for half the year, so there is nothing to correct and
    the correction must not invent an offset anyway."""
    assert submit_scrobbles.device_time_to_utc(WINTER_LOGGED, LONDON) == WINTER_TRUE
 def test_the_converted_time_reads_back_as_the_clock_the_device_showed():
    """The round trip, which is the property that actually matters: whatever
    the player's screen said is what Last.fm should show in local time."""
    shown = datetime.fromtimestamp(SUMMER_LOGGED, timezone.utc)
    corrected = submit_scrobbles.device_time_to_utc(SUMMER_LOGGED, LONDON)
    assert datetime.fromtimestamp(corrected, LONDON).strftime("%Y-%m-%d %H:%M") == (
        shown.strftime("%Y-%m-%d %H:%M")
    )
 def test_one_log_spanning_a_clock_change_converts_each_side_separately():
    """A week's listening either side of the October change carries two
    different offsets. Correcting the log by a single figure would put half of
    it an hour out, which is why the offset is resolved per play."""
    before, after = 1792888200, 1792899000  # 2026-10-25, 00:30 BST and 03:30 GMT
    assert submit_scrobbles.device_time_to_utc(before, LONDON) == before - 3600
    assert submit_scrobbles.device_time_to_utc(after, LONDON) == after
 def test_a_playback_log_is_corrected_before_submission(monkeypatch):
    monkeypatch.setattr(Path, "is_file", lambda self: True)
    log = f"{SUMMER_LOGGED}:180000:245000:/Music/Pendulum/Immersion/01.mp3\n"
    played = submit_scrobbles.plays_from_playback_log(
        log, "/Music", "/mnt/mirror", runner=tags_of(), zone=LONDON
    ).played
    assert played[0]["timestamp"] == str(SUMMER_TRUE)
 def test_a_scrobbler_log_claiming_utc_is_not_corrected_twice():
    """The format's header exists for exactly this. A client that already did
    the conversion says so, and correcting it again would break it."""
    log = LOG.replace("#TZ/UNKNOWN", "#TZ/UTC")
    played, _, _ = submit_scrobbles.parse_log(log, LONDON)
    assert [entry["timestamp"] for entry in played] == ["1700000100", "1700000300"]
 def test_a_scrobbler_log_declaring_unknown_is_corrected():
    """Rockbox writes UNKNOWN, meaning local wall clock, so the times are ours
    to fix."""
    played, _, _ = submit_scrobbles.parse_log(LOG, LONDON)
    # 1700000100 and 1700000300 are November, when London is on UTC anyway;
    # the point is that the header did not exempt them from being looked at.
    assert submit_scrobbles.declares_utc(LOG) is False
    assert [entry["timestamp"] for entry in played] == ["1700000100", "1700000300"]
 def test_the_system_zone_keeps_its_whole_name(monkeypatch, tmp_path):
    """/etc/localtime points into the tzdata tree. Taking only the last
    component gives "London", which no database holds -- and the fallback for
    an unknown name is a fixed offset, which is wrong for half the year. The
    region has to survive."""
    zoneinfo_dir = tmp_path / "share" / "zoneinfo" / "Europe"
    zoneinfo_dir.mkdir(parents=True)
    (zoneinfo_dir / "London").write_bytes(b"TZif")
    monkeypatch.setattr(
        submit_scrobbles.Path,
        "read_text",
        lambda self, **kw: (_ for _ in ()).throw(OSError),
    )
    monkeypatch.setattr(
        submit_scrobbles.Path,
        "resolve",
        lambda self: zoneinfo_dir / "London",
    )
    assert submit_scrobbles.system_zone_name() == "Europe/London"
 def test_a_named_zone_beats_the_machine_default():
    """The player may not be in the same place as the laptop."""
    assert submit_scrobbles.device_zone("Asia/Tokyo") == ZoneInfo("Asia/Tokyo")
 def test_plays_dated_after_now_are_reported():
    """A clock never put forward, or the wrong zone, produces plays that have
    not happened yet. Last.fm cannot tell those from real ones."""
    now = 1787835600
    played = [
        {"timestamp": str(now - 60)},
        {"timestamp": str(now + 3600)},
    ]
    ahead = submit_scrobbles.future_plays(played, now=now)
    assert [entry["timestamp"] for entry in ahead] == [str(now + 3600)]
 def test_a_clock_a_minute_fast_is_not_reported():
    """Devices drift. Only an offset large enough to be a zone error matters."""
    now = 1787835600
    assert submit_scrobbles.future_plays([{"timestamp": str(now + 60)}], now=now) == []
 def test_playback_logs_are_found_including_rotations(tmp_path):
    """Rockbox rotates the log once it passes half a megabyte."""
    rockbox = tmp_path / ".rockbox"
@@ -5,7 +5,6 @@ protecting against emptying the wrong directory -- a mistake that does not
 announce itself.
 """
 import os
 import shutil
 import subprocess
 from pathlib import Path
@@ -347,88 +346,3 @@ def test_counting_can_be_asked_for(mirror, tmp_path):
    assert result.returncode == 0, result.stderr
    assert "files to copy" in result.stderr
 # A ReplayGain re-level rewrites a track's tags in the padding the previous
 # write left behind, so neither the size nor the mtime changes -- and those are
 # the two things rsync's quick check compares.
 def stale_copy(mirror, destination, relative, current, previous):
    """Put a file on the device that differs only in content from the mirror's."""
    source = mirror / relative
    source.parent.mkdir(parents=True, exist_ok=True)
    source.write_bytes(current)
    device = destination / relative
    device.parent.mkdir(parents=True, exist_ok=True)
    device.write_bytes(previous)
    os.utime(device, (source.stat().st_atime, source.stat().st_mtime))
    return device
 def test_a_tag_only_change_reaches_the_device_with_the_track_that_caused_it(
    mirror, tmp_path
 ):
    """The new track is visible to rsync; its re-levelled sibling is not, and
    would otherwise keep the old album gain on the device forever."""
    destination = tmp_path / "dest"
    destination.mkdir()
    sibling = stale_copy(mirror, destination, "Album/sibling.mp3", b"NEW", b"OLD")
    result = run("-f", "-S", "-U", str(mirror), str(destination))
    assert result.returncode == 0, result.stderr
    assert sibling.read_bytes() == b"NEW"
    assert (destination / "Album" / "track.mp3").is_file()
 def test_a_deletion_also_relevels_what_is_left_behind(mirror, tmp_path):
    destination = tmp_path / "dest"
    destination.mkdir()
    sibling = stale_copy(mirror, destination, "Album/sibling.mp3", b"NEW", b"OLD")
    (destination / "Album" / "gone.mp3").write_bytes(b"old")
    result = run("-f", "-S", "-U", str(mirror), str(destination))
    assert result.returncode == 0, result.stderr
    assert sibling.read_bytes() == b"NEW"
    assert not (destination / "Album" / "gone.mp3").exists()
 def test_an_untouched_album_is_not_copied_again(mirror, tmp_path):
    """The second pass is scoped to albums that changed. An album whose file
    set is the same is left where it is, which is the whole point of not
    running --ignore-times over the library."""
    destination = tmp_path / "dest"
    destination.mkdir()
    quiet = stale_copy(mirror, destination, "Quiet/only.mp3", b"NEW", b"OLD")
    (destination / "Album").mkdir()
    shutil.copy2(mirror / "Album" / "track.mp3", destination / "Album" / "track.mp3")
    result = run("-f", "-S", "-U", str(mirror), str(destination))
    assert result.returncode == 0, result.stderr
    assert quiet.read_bytes() == b"OLD"
 def test_a_dry_run_says_how_many_extra_tracks_are_involved(mirror, tmp_path):
    destination = tmp_path / "dest"
    destination.mkdir()
    stale_copy(mirror, destination, "Album/sibling.mp3", b"NEW", b"OLD")
    result = run("-f", "-S", "-U", "-n", str(mirror), str(destination))
    assert result.returncode == 0, result.stderr
    assert "and 1 more in those albums" in result.stderr
 def test_a_first_sync_does_not_copy_anything_twice(mirror, tmp_path):
    """Everything is transferred by the main pass, so there is nothing left for
    the second one and it must not announce itself."""
    destination = tmp_path / "dest"
    destination.mkdir()
    result = run("-f", "-S", "-U", str(mirror), str(destination))
    assert result.returncode == 0, result.stderr
    assert "re-levelled" not in result.stderr
@@ -1,128 +0,0 @@
 """A ReplayGain re-level rewrites a track's tags without changing its size or
 its mtime, which is precisely the pair rsync's quick check compares. These
 cover the list that is fed back to rsync to copy those tracks anyway."""
 import io
 import subprocess
 import sys
 from pathlib import Path
 sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "tools"))
 import touched_albums  # noqa: E402
 def album(root, name, *tracks):
    directory = root / name
    directory.mkdir(parents=True, exist_ok=True)
    for track in tracks:
        (directory / track).write_bytes(b"x")
    return directory
 def listing(mirror, *lines):
    return touched_albums.remaining(mirror, *touched_albums.touched(list(lines)))
 def test_the_siblings_of_a_new_track_are_listed(tmp_path):
    album(tmp_path, "Artist/Album", "01.mp3", "02.mp3", "03.mp3")
    assert listing(tmp_path, "4096 Artist/Album/03.mp3") == [
        "Artist/Album/01.mp3",
        "Artist/Album/02.mp3",
    ]
 def test_a_track_rsync_just_copied_is_not_copied_twice(tmp_path):
    """A quality upgrade replaces every track on the record. Listing them again
    would send the album across twice."""
    album(tmp_path, "Artist/Album", "01.mp3", "02.mp3")
    assert listing(tmp_path, "4096 Artist/Album/01.mp3", "4096 Artist/Album/02.mp3") == []
 def test_a_deletion_relevels_what_is_left(tmp_path):
    album(tmp_path, "Artist/Album", "01.mp3", "02.mp3")
    assert listing(tmp_path, "deleting Artist/Album/03.mp3") == [
        "Artist/Album/01.mp3",
        "Artist/Album/02.mp3",
    ]
 def test_an_album_deleted_outright_lists_nothing(tmp_path):
    assert listing(tmp_path, "deleting Artist/Gone/01.mp3") == []
 def test_untouched_albums_are_left_alone(tmp_path):
    album(tmp_path, "Artist/Changed", "01.mp3", "02.mp3")
    album(tmp_path, "Artist/Quiet", "01.mp3", "02.mp3")
    assert listing(tmp_path, "4096 Artist/Changed/01.mp3") == ["Artist/Changed/02.mp3"]
 def test_a_replaced_cover_is_not_an_album_change(tmp_path):
    """Only a track can change an album's gains, and covers are replaced often
    enough that treating one as a re-level would copy records for nothing."""
    album(tmp_path, "Artist/Album", "01.mp3", "02.mp3")
    assert listing(tmp_path, "17408 Artist/Album/cover.jpg") == []
 def test_directories_are_not_mistaken_for_tracks(tmp_path):
    album(tmp_path, "Artist/Album", "01.mp3")
    assert listing(tmp_path, "4096 Artist/Album/", "deleting Artist/Old/") == []
 def test_rsync_talking_to_the_operator_is_not_a_path(tmp_path):
    album(tmp_path, "Artist/Album", "01.mp3", "02.mp3")
    assert (
        listing(
            tmp_path,
            "sending incremental file list",
            "",
            "sent 1,234 bytes  received 56 bytes  2,580.00 bytes/sec",
            "total size is 7,890  speedup is 6.12",
        )
        == []
    )
 def test_a_track_at_the_mirror_root_does_not_pull_in_the_whole_tree(tmp_path):
    """Nothing writes a mirror this way, but the directory of a root-level file
    is the root, and recursing from there would be the whole library."""
    (tmp_path / "loose.mp3").write_bytes(b"x")
    (tmp_path / "other.mp3").write_bytes(b"x")
    album(tmp_path, "Artist/Album", "01.mp3")
    assert listing(tmp_path, "4096 loose.mp3") == ["other.mp3"]
 def test_the_paths_are_written_one_per_line(tmp_path):
    """They are fed straight back to rsync as --files-from."""
    album(tmp_path, "Artist/Album", "01.mp3", "02.mp3")
    out = io.StringIO()
    touched_albums.main(
        ["--mirror", str(tmp_path)],
        stream=["4096 Artist/Album/01.mp3"],
        out=out,
    )
    assert out.getvalue() == "Artist/Album/02.mp3\n"
 def test_it_runs_as_a_script(tmp_path):
    album(tmp_path, "Artist/Album", "01.mp3", "02.mp3")
    completed = subprocess.run(
        [sys.executable, touched_albums.__file__, "--mirror", str(tmp_path)],
        input="4096 Artist/Album/01.mp3\n",
        capture_output=True,
        text=True,
    )
    assert completed.returncode == 0
    assert completed.stdout == "Artist/Album/02.mp3\n"
@@ -21,7 +21,9 @@ import urllib.error
 import urllib.parse
 import urllib.request
 from dataclasses import dataclass
 from datetime import datetime, timezone
 from pathlib import Path
 from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
 API_ROOT = "https://ws.audioscrobbler.com/2.0/"
@@ -48,6 +50,11 @@ LISTENED_FRACTION = 0.5
 # Rockbox logs ticks in milliseconds instead, which is not a date.
 EARLIEST_PLAUSIBLE = 1_000_000_000
 # How far ahead of now a converted play may sit before it is reported. Some
 # slack absorbs a device clock drifting by a minute or two; an hour out means
 # the zone is wrong or the clock was never put forward.
 FUTURE_TOLERANCE_SECONDS = 300
 SESSION_FILE = Path(
    os.getenv("XDG_CONFIG_HOME", Path.home() / ".config")
 ) / "music-mirror" / "lastfm.json"
@@ -57,14 +64,117 @@ class LastfmError(Exception):
    """A Last.fm request that failed."""
-def parse_log(text):
+def system_zone_name():
    """Return this machine's IANA zone name, or "" if nothing states it.
    The name has to come out whole. /etc/localtime is a symlink into the
    tzdata tree, so the part after `zoneinfo/` is the name -- taking only the
    last component yields "London", which no database has, and falls back to a
    fixed offset that would then be wrong for half the year.
    """
    try:
        named = Path("/etc/timezone").read_text(encoding="utf-8").strip()
        if named:
            return named
    except OSError:
        pass
    try:
        parts = Path("/etc/localtime").resolve().parts
    except OSError:
        return ""
    if "zoneinfo" in parts:
        return "/".join(parts[len(parts) - parts[::-1].index("zoneinfo"):])
    return ""
 def device_zone(name=None):
    """Return the zone the device's clock is keeping.
    Rockbox has no concept of a timezone, so its clock is set to local wall
    time and the zone has to be supplied from outside. Defaulting to this
    machine's zone is right whenever the player and the laptop are in the same
    place, which for a device synced by cable they are.
    """
    if name:
        return ZoneInfo(name)
    for candidate in (os.getenv("TZ"), system_zone_name()):
        if not candidate:
            continue
        try:
            return ZoneInfo(candidate)
        except (ZoneInfoNotFoundError, ValueError):
            continue
    # Nothing named the zone, so the offset cannot be resolved per play: this
    # is today's offset applied to every timestamp, which is wrong either side
    # of a daylight saving change. Still better than pretending it logged UTC.
    print(
        "warning: no IANA timezone found for this machine; using its current"
        " offset for every play. Pass --device-timezone to fix older plays"
        " across a daylight saving change.",
        file=sys.stderr,
    )
    return datetime.now().astimezone().tzinfo
 def device_time_to_utc(stamp, zone):
    """Return the true UTC epoch of a timestamp Rockbox wrote.
    Rockbox builds its timestamps with `mktime(get_time())`, and its mktime
    (firmware/libc/mktime.c) is plain calendar arithmetic with no zone applied.
    Fed the RTC's local fields it yields local-wall-clock-as-if-UTC, so the
    number is ahead of real UTC by whatever the offset was. Its own scrobbler
    plugin admits this by writing `#TZ/UNKNOWN`, leaving the correction here.
    Decoding the number back into those fields and reinterpreting them in the
    device's zone recovers the instant, and does so per play, so a log
    straddling a daylight-saving change converts each side by its own offset.
    """
    fields = datetime.fromtimestamp(stamp, timezone.utc).replace(tzinfo=zone)
    return int(fields.timestamp())
 def future_plays(played, now=None):
    """Return the plays timestamped later than now, which cannot have happened.
    A device clock left on the wrong offset, or never adjusted across a
    daylight-saving change, shows up here. Last.fm has no way to tell such a
    scrobble from a real one, so it is worth saying out loud.
    """
    now = time.time() if now is None else now
    return [
        entry for entry in played
        if int(entry["timestamp"]) > now + FUTURE_TOLERANCE_SECONDS
    ]
 def declares_utc(text):
    """Whether an AUDIOSCROBBLER log says its timestamps are already UTC.
    The format's header carries `#TZ/UTC` or `#TZ/UNKNOWN`, and its spec is
    explicit that a device may only claim UTC if it converted. Rockbox writes
    UNKNOWN, meaning the times are local wall clock and want correcting; a log
    from anything that claims UTC must be left alone.
    """
    for line in text.splitlines():
        if not line.startswith("#"):
            break
        if line.strip().upper().startswith("#TZ/"):
            return line.strip().upper() == "#TZ/UTC"
    return False
 def parse_log(text, zone=None):
    """Return the listened tracks in an AUDIOSCROBBLER log, oldest first.
    Fields are artist, album, title, track number, length, rating, timestamp
    and MusicBrainz id. Rockbox converts any tab inside a field to a space
    before writing, so splitting on tabs is safe.
    Timestamps are corrected from the device's local wall clock to UTC unless
    the log's own header claims it did that already.
    """
    played, skipped, timeless = [], 0, 0
    convert = zone is not None and not declares_utc(text)
    for line in text.splitlines():
        if not line or line.startswith("#"):
            continue
@@ -85,6 +195,8 @@ def parse_log(text):
            continue
        if not artist or not title:
            continue
        if convert:
            when = device_time_to_utc(when, zone)
        played.append(
            {
                "artist": artist,
@@ -173,11 +285,13 @@ class Conversion:
            self.retain = []
-def plays_from_playback_log(text, device_prefix, mirror, runner=None):
+def plays_from_playback_log(text, device_prefix, mirror, runner=None, zone=None):
    """Return the conversion of a playback log.
    Skips are decided by the same fraction the on-device plugin uses, so the
-    two never disagree about what counted as a play.
+    two never disagree about what counted as a play. Timestamps are corrected
    from the device's wall clock to UTC; the core log has no header to say so,
    but it is written the same way the plugin's UNKNOWN times are.
    """
    result = Conversion(played=[])
    for line in text.splitlines():
@@ -209,7 +323,9 @@ def plays_from_playback_log(text, device_prefix, mirror, runner=None):
                "album": tags.get("album", ""),
                "trackNumber": (tags.get("track") or "").split("/")[0],
                "duration": str(length // 1000) if length > 0 else "",
-                "timestamp": str(stamp),
+                "timestamp": str(
                    device_time_to_utc(stamp, zone) if zone is not None else stamp
                ),
                "mbid": tags.get("musicbrainz_trackid", ""),
                "line": line,
            }
@@ -350,6 +466,14 @@ def main(argv=None, transport=http_post):
        help="where the music sits on the device, stripped when mapping a logged"
        " path back onto the mirror",
    )
    parser.add_argument(
        "--device-timezone",
        default=os.getenv("ROCKBOX_TIMEZONE"),
        help="the zone the player's clock is set to, as an IANA name such as"
        " Europe/London. Rockbox keeps local wall time and cannot record an"
        " offset, so its timestamps need this to become the UTC Last.fm wants."
        " Defaults to this machine's zone",
    )
    parser.add_argument("--api-key", default=os.getenv("LASTFM_API_KEY"))
    parser.add_argument("--api-secret", default=os.getenv("LASTFM_API_SECRET"))
    parser.add_argument("--dry-run", action="store_true", help="parse and report only")
@@ -358,6 +482,12 @@ def main(argv=None, transport=http_post):
    )
    args = parser.parse_args(argv)
    try:
        zone = device_zone(args.device_timezone)
    except (ZoneInfoNotFoundError, ValueError) as error:
        print(f"unknown timezone {args.device_timezone!r}: {error}", file=sys.stderr)
        return 2
    target = Path(args.device)
    log = target if target.is_file() else find_log(target)
    logs = []
@@ -365,7 +495,7 @@ def main(argv=None, transport=http_post):
    conversion = None
    if log is not None:
        played, skipped, timeless = parse_log(
-            log.read_text(encoding="utf-8", errors="replace")
+            log.read_text(encoding="utf-8", errors="replace"), zone
        )
        unresolved = 0
        logs = [log]
@@ -380,7 +510,9 @@ def main(argv=None, transport=http_post):
        text = "\n".join(
            path.read_text(encoding="utf-8", errors="replace") for path in logs
        )
-        conversion = plays_from_playback_log(text, args.device_prefix, args.mirror)
+        conversion = plays_from_playback_log(
            text, args.device_prefix, args.mirror, zone=zone
        )
        played = conversion.played
        skipped, unresolved, timeless = (
            conversion.skipped,
@@ -413,9 +545,24 @@ def main(argv=None, transport=http_post):
        )
    if not played:
        return 0
    ahead = future_plays(played)
    if ahead:
        newest = int(ahead[-1]["timestamp"]) - int(time.time())
        print(
            f"  {len(ahead)} plays are timestamped up to {newest // 60} minutes in"
            f" the future, converting from {zone}. Either the player's clock is"
            " wrong or that is not the zone it is set to.",
            file=sys.stderr,
        )
    if args.dry_run:
        for entry in played[:20]:
-            print(f"{entry['timestamp']}\t{entry['artist']}\t{entry['track']}")
+            when = datetime.fromtimestamp(int(entry["timestamp"]), zone)
            print(
                f"{entry['timestamp']}\t{when:%Y-%m-%d %H:%M %Z}"
                f"\t{entry['artist']}\t{entry['track']}"
            )
        return 0
    if not args.api_key or not args.api_secret:
@@ -41,6 +41,12 @@ Submitting scrobbles needs LASTFM_API_KEY and LASTFM_API_SECRET; it is skipped
 with a note when they are unset. Scrobbling is a write method and needs the
 secret, unlike the read-only calls elsewhere in these projects.
 Rockbox has no notion of a timezone: its clock holds local wall time and its
 logs record that, not UTC. Set ROCKBOX_TIMEZONE to the zone the player's clock
 is keeping (an IANA name, such as Europe/London) if it differs from this
 machine's, which is otherwise assumed. Getting it wrong shifts every scrobble
 by the difference.
 The mirror is the directory holding the artist folders. The destination is
 where those folders should end up on the device -- not the card root, unless
 that is genuinely where you want them:
@@ -56,10 +62,6 @@ The destination must be on a mounted FAT filesystem. That check is also what
 catches an unmounted device: /media/IPOD/Music then resolves to the host's own
 root filesystem, and this refuses to empty that.
 An album that gained or lost a track is copied again in full afterwards. Its
 surviving tracks have had their ReplayGain tags rewritten in place, which
 changes neither their size nor their mtime, so the main pass cannot see them.
 Progress is one line that rewrites itself, showing the album currently going
 across, how far through the transfer is, the rate, and an estimate of what is
 left. Working the totals out first means a second pass over the tree, which is
@@ -162,6 +164,11 @@ if $scrobble; then
    else
        scrobble_options=()
        $dry_run && scrobble_options+=(--dry-run)
        # Rockbox keeps local wall time with no notion of a zone, so its
        # timestamps are not the UTC Last.fm expects. Naming the zone the
        # player's clock is set to lets them be corrected.
        [ -n "${ROCKBOX_TIMEZONE:-}" ] &&
            scrobble_options+=(--device-timezone "$ROCKBOX_TIMEZONE")
        # --mirror lets it convert Rockbox's own playback.log, so the on-device
        # scrobbler plugin never has to be run. The device root, not the music
        # directory: the logs live in .rockbox.
@@ -197,23 +204,8 @@ done
 printf 'sync-to-ipod: %s -> %s\n' "$mirror" "$destination" >&2
 # Both passes below feed their file list through touched_albums.py, which reads
 # rsync's own report of what it moved. --files-from separates paths by newline,
 # so a filename containing one would be read as two -- which is a path FAT32
 # will not take either, and check_fat32.py above has already refused the run
 # unless -f was given to skip it.
 changed=$(mktemp)
 relevelled=$(mktemp)
 trap 'rm -f "$changed" "$relevelled"' EXIT
 if $dry_run; then
-    rsync "${options[@]}" --dry-run --verbose --out-format='%l %n' \
+    rsync "${options[@]}" --dry-run --verbose "$mirror/" "$destination/"
        "$mirror/" "$destination/" | tee "$changed"
    also=$(python3 "$here/touched_albums.py" --mirror "$mirror" <"$changed" | wc -l)
    if [ "$also" -gt 0 ]; then
        printf 'sync-to-ipod: and %s more in those albums, whose ReplayGain tags\n' "$also" >&2
        printf 'sync-to-ipod: change without changing their size or their mtime\n' >&2
    fi
    printf 'sync-to-ipod: dry run, nothing was written\n' >&2
    exit 0
 fi
@@ -275,33 +267,10 @@ interrupted() {
 trap interrupted INT TERM
 rsync "${options[@]}" --out-format='%l %n' "$mirror/" "$destination/" |
    tee "$changed" |
    python3 "$here/rsync_progress.py" --total "$total" --bytes "$total_bytes"
 status=${PIPESTATUS[0]}
 [ "$status" -eq 0 ] || die "rsync exited $status"
 # A ReplayGain album gain belongs to the whole record, so a track arriving or
 # leaving rewrites the tags on all of its siblings. rsgain fits the new values
 # into the padding its previous write left behind, which changes neither the
 # size nor the mtime -- the only two things the pass above compares. Those
 # tracks are invisible to it, and the device would keep the old gains.
 #
 # The track that arrived or left is visible, though. So every album the pass
 # touched has the rest of its tracks copied again, with --ignore-times to
 # defeat the same quick check. No --delete: the pass above has already settled
 # what should be on the device, and --delete aimed at an explicit file list
 # does not mean what it looks like it means.
 python3 "$here/touched_albums.py" --mirror "$mirror" <"$changed" >"$relevelled"
 if [ -s "$relevelled" ]; then
    also=$(wc -l <"$relevelled")
    printf 'sync-to-ipod: re-copying %s tracks whose album was re-levelled\n' "$also" >&2
    rsync --times --modify-window=2 --whole-file --omit-dir-times --ignore-times \
        --files-from="$relevelled" --out-format='%l %n' "$mirror/" "$destination/" |
        python3 "$here/rsync_progress.py" --total "$also"
    status=${PIPESTATUS[0]}
    [ "$status" -eq 0 ] || die "the re-levelled tracks failed to copy: rsync exited $status"
 fi
 # Rockbox reads its database from .tcd files in .rockbox. Building them here
 # rather than on the device is not just faster: the on-device commit sorts the
 # whole index in whatever memory it can scrape together, and on a large library
@@ -1,97 +0,0 @@
 #!/usr/bin/env python3
 """List the tracks a sync has to copy again because their album was re-levelled.
 Reads rsync's `--out-format='%l %n'` output on stdin and writes mirror-relative
 paths on stdout, one per line, for feeding straight back to rsync as
 `--files-from`.
 The problem it solves: a ReplayGain album gain is a property of every track on
 the record, so one track arriving or leaving changes the tags on all of its
 siblings. rsgain writes the new values into the padding its previous write left
 behind, which leaves both the file's size and its mtime untouched -- and size
 and mtime are exactly what rsync's quick check compares. It sees nothing to do,
 and the device keeps the old gains.
 What rsync always can see is the track that arrived or left. So any album it
 touched has the rest of its tracks copied again, and nothing else does.
 Tracks rsync has already dealt with are left out of the list. After a quality
 upgrade that replaces every track on a record, listing them again would send
 the whole album twice.
 """
 import argparse
 import re
 import sys
 from pathlib import Path
 # Under --out-format='%l %n' a transfer is reported as the size, a space and
 # the path. A removal is reported as "deleting <path>" whatever the format is.
 # Everything else on the stream is rsync talking to the operator -- the file
 # list preamble, the byte totals -- and is not a path.
 TRANSFER = re.compile(r"^(\d+) (.+)$")
 DELETION = re.compile(r"^deleting (.+)$")
 # What the mirror is made of, and so the only thing worth copying again. A
 # cover is not rewritten by a re-level.
 MIRROR_SUFFIX = ".mp3"
 def touched(lines):
    """Return the paths rsync transferred and the directories it changed."""
    transferred = set()
    directories = set()
    for line in lines:
        line = line.rstrip("\n")
        deletion = DELETION.match(line)
        transfer = None if deletion else TRANSFER.match(line)
        if deletion:
            path = deletion.group(1)
        elif transfer:
            path = transfer.group(2)
        else:
            continue
        # Only a track can change an album's gains. rsync reports the
        # directories it creates and removes too, and a cover replaced on its
        # own is no reason to send the record again.
        if not path.endswith(MIRROR_SUFFIX):
            continue
        if transfer:
            transferred.add(path)
        directories.add(path.rpartition("/")[0])
    return transferred, directories
 def remaining(mirror, transferred, directories):
    """Return the tracks in those directories that rsync has not just copied."""
    paths = []
    for directory in sorted(directories):
        album = mirror / directory if directory else mirror
        if not album.is_dir():
            # Removed along with the last of its tracks. Nothing to copy.
            continue
        for track in sorted(album.glob(f"*{MIRROR_SUFFIX}")):
            relative = f"{directory}/{track.name}" if directory else track.name
            if relative not in transferred:
                paths.append(relative)
    return paths
 def main(argv=None, stream=None, out=None):
    parser = argparse.ArgumentParser(
        prog="touched_albums.py",
        description="List the tracks to copy again after an album was re-levelled.",
    )
    parser.add_argument("--mirror", required=True, type=Path, help="root of the MP3 mirror")
    args = parser.parse_args(argv)
    transferred, directories = touched(stream if stream is not None else sys.stdin)
    for path in remaining(args.mirror, transferred, directories):
        print(path, file=out if out is not None else sys.stdout)
    return 0
 if __name__ == "__main__":
    sys.exit(main())