Emma Thorpe
3141f7ca87
Build and publish container / build (pull_request) Successful in 2m18s
Two changes for playing the mirror on a Rockbox iPod, where the device is FAT32 and Rockbox reads a plain directory tree rather than a database. --fat32-safe names mirror files acceptably: the reserved characters and control characters become underscores, trailing dots and spaces are stripped because FAT eats them silently and the name then round-trips as a different one, and a component left empty becomes an underscore. Names differing only in case are detected as collisions, since two files here are one file there and the second would silently overwrite the first. "Kick Out the Epic Motherf**ker" is a real example from a real library, and without this it simply never arrives. Off by default. It renames files, and that should be a decision rather than a surprise on somebody's next pass. Turning it on does not re-encode anything. Every track whose name held a reserved character changes path, and encoding those again would be hours of work producing files that already exist byte for byte, so the run moves them instead and logs each one. Prune then finds nothing left behind. Album art is now also copied into the mirror as cover.jpg beside the tracks. Rockbox searches the filesystem for art -- cover.jpg, folder.jpg and the rest, in the track's directory or its parent -- and that search never looks at the picture embedded in the tag, so a mirror that only embeds art displays none of it on the device. Embedding continues for the Apple firmware; both are now satisfied. A cover whose tracks have all been pruned is removed too, or its directory would never look empty and never go. Adds tools/check_fat32.py, which reports unacceptable paths before a copy rather than during one: rsync reports them too, but scattered through fifty thousand files where they are easy to lose. It exits non-zero so it can gate a script. The README documents the rsync invocation, including why --modify-window=2 is required against FAT and why Rhythmbox must be kept out of the transfer -- rb_ipod_helpers_is_ipod() reads access-protocols from media-player-info and returns true on the USB id alone, without looking at the filesystem, so removing iPod_Control changes nothing.
94 lines
3.2 KiB
Python
Executable File
94 lines
3.2 KiB
Python
Executable File
#!/usr/bin/env python3
|
|
"""Report paths a FAT32 device will not accept, before copying rather than during.
|
|
|
|
Run this against the mirror before an rsync to a Rockbox iPod. rsync will
|
|
report the failures too, but scattered through a run of fifty thousand files,
|
|
where they are easy to lose.
|
|
|
|
Checks the four ways a name fails on FAT32: reserved characters, trailing dots
|
|
or spaces that FAT silently eats, components longer than 255 characters, and
|
|
names that differ only in case -- two files here, one file there, and the
|
|
second silently overwrites the first.
|
|
"""
|
|
|
|
import argparse
|
|
import os
|
|
import re
|
|
import sys
|
|
import unicodedata
|
|
from collections import defaultdict
|
|
from pathlib import Path
|
|
|
|
RESERVED = re.compile(r'[<>:"\\|?*\x00-\x1f]')
|
|
COMPONENT_LIMIT = 255
|
|
# Rockbox builds paths into a fixed buffer; long trees fail on the device even
|
|
# when every individual component is legal.
|
|
PATH_LIMIT = 260
|
|
|
|
|
|
def problems_with(relative):
|
|
"""Return every reason this relative path is unfit for FAT32."""
|
|
found = []
|
|
for part in relative.parts:
|
|
if RESERVED.search(part):
|
|
found.append(f"reserved character in {part!r}")
|
|
if part != part.rstrip(". "):
|
|
found.append(f"trailing dot or space in {part!r}")
|
|
if len(part) > COMPONENT_LIMIT:
|
|
found.append(f"component of {len(part)} characters")
|
|
if len(str(relative)) > PATH_LIMIT:
|
|
found.append(f"path of {len(str(relative))} characters")
|
|
return found
|
|
|
|
|
|
def walk(root):
|
|
"""Yield every file below a root, as a path relative to it."""
|
|
for base, _, names in os.walk(root):
|
|
for name in names:
|
|
yield Path(os.path.join(base, name)).relative_to(root)
|
|
|
|
|
|
def main(argv=None):
|
|
parser = argparse.ArgumentParser(description=__doc__)
|
|
parser.add_argument("root", help="directory to check, e.g. the mirror")
|
|
parser.add_argument("--limit", type=int, default=0, help="show at most this many")
|
|
args = parser.parse_args(argv)
|
|
|
|
root = Path(args.root)
|
|
if not root.is_dir():
|
|
print(f"{root} is not a directory", file=sys.stderr)
|
|
return 2
|
|
|
|
faults = []
|
|
by_case = defaultdict(list)
|
|
total = 0
|
|
for relative in walk(root):
|
|
total += 1
|
|
# NFC first: the same name written by two systems is otherwise two
|
|
# different strings, and the collision check would miss it.
|
|
key = unicodedata.normalize("NFC", str(relative)).casefold()
|
|
by_case[key].append(relative)
|
|
for problem in problems_with(relative):
|
|
faults.append((relative, problem))
|
|
|
|
for relative, group in sorted(by_case.items()):
|
|
if len(group) > 1:
|
|
names = ", ".join(str(path) for path in sorted(group))
|
|
faults.append((group[0], f"collides case-insensitively with: {names}"))
|
|
|
|
for relative, problem in faults[: args.limit or None]:
|
|
print(f"{relative}\t{problem}")
|
|
|
|
print(f"\n{len(faults)} problems across {total} files", file=sys.stderr)
|
|
if faults:
|
|
print(
|
|
"Run music-mirror with --fat32-safe to have the mirror named"
|
|
" acceptably in the first place.",
|
|
file=sys.stderr,
|
|
)
|
|
return 1 if faults else 0
|
|
|
|
|
|
if __name__ == "__main__":
|
|
sys.exit(main())
|