perf: cut round trips over a network mount, and allow skipping the count
Build and publish container / build (pull_request) Successful in 2m39s

The transfer is metadata-bound rather than throughput-bound. Fifty thousand
files is fifty thousand round trips, and the counting pass added for the
percentage doubles that.

--whole-file is already implied when both ends are local paths, which an SMB or
FAT mount is, but stating it records that the delta algorithm is deliberately
unwanted here: it would read every destination file back over USB to checksum
it, in order to avoid resending an MP3 that has changed in its entirety anyway.

--omit-dir-times drops one setattr per directory. Across six thousand album
folders on a FAT card that is six thousand operations spent on timestamps
nothing reads.

-Q skips the counting pass. The percentage and the estimate are worth a second
walk of a local tree and frequently are not worth one of a network mount, so
that is now a choice rather than a fixed cost.

The README covers the part that is not an rsync flag at all: SMB defaults to a
one second attribute cache, so nearly every stat goes to the wire, twice. An
actimeo of sixty on the mount does more than any of the above, and closes most
of the gap that would otherwise argue for moving to NFS.
This commit is contained in:
Emma Thorpe
2026-08-25 12:44:53 +01:00
parent 131c80f5de
commit d6ef70922c
3 changed files with 84 additions and 11 deletions
+24
View File
@@ -185,6 +185,30 @@ is what a percentage and an estimate that mean something cost — and renders th
rest itself. Piped to a log it prints a plain line every thirty seconds
instead, with no carriage returns, and a summary at the end either way.
### Making it faster over a network mount
The transfer is metadata-bound, not throughput-bound: 49,600 files means 49,600
round trips, and the counting pass doubles that. In rough order of what it is
worth doing:
| Lever | Why |
| ----- | --- |
| Mount the source with `actimeo=60,cache=loose` | SMB defaults to a **one second** attribute cache, so nearly every `stat` goes to the wire — twice, once per pass. This is the single biggest change and it is a mount option, not an rsync flag. |
| Put the card in a reader for the first load | USB 2.0 through an iPod in disk mode is the floor for the destination. No amount of source tuning gets past it. |
| `-Q` | Skips the counting pass entirely. Costs the percentage and the estimate, saves a whole walk of the tree. |
| `--whole-file`, `--omit-dir-times` | Already set. The first stops rsync checksumming destination files it is about to overwrite whole; the second drops a setattr per directory, 6,150 of them. |
**NFS instead of SMB** is worth trying but is not the big win it looks like.
Its attribute caching defaults are far more generous than SMB's — `acregmax` of
sixty seconds against `actimeo=1` — which is precisely the gap that
`actimeo=60` closes on the mount you already have. Bulk read throughput between
the two is much of a muchness on a gigabit link. Try the mount option first; it
is one line and needs no change on the NAS.
And if the destination is the iPod rather than a card reader, none of this
matters much: the source can feed data faster than USB 2.0 through an iPod will
take it either way.
The unmount is the point of doing this in a script. FAT32 has no journal and
the device is reached through disk mode, so an interrupted write is corruption
that needs `fsck.vfat` from another machine.