perf: cut round trips over a network mount, and allow skipping the count
Build and publish container / build (pull_request) Successful in 2m39s
Build and publish container / build (pull_request) Successful in 2m39s
The transfer is metadata-bound rather than throughput-bound. Fifty thousand files is fifty thousand round trips, and the counting pass added for the percentage doubles that. --whole-file is already implied when both ends are local paths, which an SMB or FAT mount is, but stating it records that the delta algorithm is deliberately unwanted here: it would read every destination file back over USB to checksum it, in order to avoid resending an MP3 that has changed in its entirety anyway. --omit-dir-times drops one setattr per directory. Across six thousand album folders on a FAT card that is six thousand operations spent on timestamps nothing reads. -Q skips the counting pass. The percentage and the estimate are worth a second walk of a local tree and frequently are not worth one of a network mount, so that is now a choice rather than a fixed cost. The README covers the part that is not an rsync flag at all: SMB defaults to a one second attribute cache, so nearly every stat goes to the wire, twice. An actimeo of sixty on the mount does more than any of the above, and closes most of the gap that would otherwise argue for moving to NFS.
This commit is contained in:
@@ -174,3 +174,33 @@ def test_the_help_says_how_to_reach_and_leave_disk_mode():
|
||||
|
||||
assert "Menu+Select" in help_text
|
||||
assert "holding Play" in help_text
|
||||
|
||||
|
||||
def test_quick_mode_skips_the_counting_pass(mirror, tmp_path):
|
||||
"""Over SMB the walk is the expensive part, and doing it twice for a
|
||||
percentage is not always the trade you want."""
|
||||
destination = tmp_path / "dest"
|
||||
destination.mkdir()
|
||||
|
||||
result = run("-f", "-S", "-U", "-Q", str(mirror), str(destination))
|
||||
|
||||
assert result.returncode == 0, result.stderr
|
||||
assert "skipping the count" in result.stderr
|
||||
assert "files to copy" not in result.stderr
|
||||
assert (destination / "Album" / "track.mp3").is_file()
|
||||
|
||||
|
||||
def test_the_delta_algorithm_is_disabled(mirror, tmp_path):
|
||||
"""It would read every destination file back over USB to checksum it, to
|
||||
avoid resending an MP3 that has changed in its entirety anyway."""
|
||||
script = SCRIPT.read_text()
|
||||
|
||||
assert "--whole-file" in script
|
||||
|
||||
|
||||
def test_directory_timestamps_are_not_set(mirror, tmp_path):
|
||||
"""One setattr round trip per directory, across six thousand albums, to set
|
||||
timestamps nothing reads."""
|
||||
script = SCRIPT.read_text()
|
||||
|
||||
assert "--omit-dir-times" in script
|
||||
|
||||
Reference in New Issue
Block a user