fix: stop counting by default; the pass costs more than the transfer
Build and publish container / build (pull_request) Successful in 3m32s
Build and publish container / build (pull_request) Successful in 3m32s
The counting pass was added so the progress line could show a percentage and an estimate, and on a real card it turned out to dominate the run. Measured against the device: reading a track from the SMB mirror ran at 35 MB/s and writing to the card at 21 MB/s, while the sync itself managed tens of kilobytes per second. Neither end was slow. The cost was traversing fifty thousand files across six thousand directories on FAT, and the counting pass does that a second time, comparing both trees in full exactly as the transfer does. Counting is now opt-in behind -P. Without it the progress line still shows the running count, the transfer rate and the album in flight; the percentage and the estimate are what needed the extra walk, and they were the least useful part of the display. That the fix for "it looks hung" was itself making it slow is the sort of thing only measuring catches. The line still answers the question it was added for -- whether anything is happening -- without paying for the part that merely made it prettier.
This commit is contained in:
+13
-8
@@ -22,8 +22,9 @@ usage() {
|
||||
usage: sync-to-ipod.sh [options] <mirror> <destination>
|
||||
|
||||
-n dry run; show what would change and touch nothing
|
||||
-Q skip the counting pass; no percentage or estimate, but one less walk
|
||||
of the source tree, which over SMB is the expensive part
|
||||
-P count what needs copying first, so progress can show a percentage and
|
||||
an estimate. Costs a second full traversal of both trees, which on a
|
||||
FAT card of fifty thousand files is slower than the transfer itself
|
||||
-f copy even if the FAT32 check finds unacceptable paths
|
||||
-S skip submitting the Rockbox scrobbler log to Last.fm
|
||||
-B skip rebuilding the Rockbox database
|
||||
@@ -68,7 +69,7 @@ USAGE
|
||||
}
|
||||
|
||||
dry_run=false
|
||||
quick=false
|
||||
counting=false
|
||||
force=false
|
||||
unmount=true
|
||||
scrobble=true
|
||||
@@ -76,10 +77,10 @@ database=true
|
||||
for argument in "$@"; do
|
||||
[ "$argument" = "--help" ] && usage help
|
||||
done
|
||||
while getopts ":nQfSBUh" option; do
|
||||
while getopts ":nPfSBUh" option; do
|
||||
case "$option" in
|
||||
n) dry_run=true ;;
|
||||
Q) quick=true ;;
|
||||
P) counting=true ;;
|
||||
f) force=true ;;
|
||||
S) scrobble=false ;;
|
||||
B) database=false ;;
|
||||
@@ -198,11 +199,15 @@ fi
|
||||
# thousand files over USB is minutes of apparent hang. Counting first costs a
|
||||
# second pass over the tree but means the transfer can show a real percentage
|
||||
# rather than a number that grows as rsync discovers more work.
|
||||
# Counting is opt-in because it is not cheap. It walks and compares both trees
|
||||
# in full, exactly as the transfer does, and on a FAT card holding fifty
|
||||
# thousand files that traversal costs more than moving the data. Without it the
|
||||
# progress line still shows the running count, the rate and the album in
|
||||
# flight; only the percentage and the estimate are lost, and those were the
|
||||
# least useful part of it.
|
||||
total=0
|
||||
total_bytes=0
|
||||
if $quick; then
|
||||
printf 'sync-to-ipod: skipping the count; no percentage or estimate\n' >&2
|
||||
else
|
||||
if $counting; then
|
||||
printf 'sync-to-ipod: working out what needs copying...\n' >&2
|
||||
# %l is the file's size, which is what makes an estimate possible.
|
||||
# Directories are dropped: rsync reports those too, with an inode size that
|
||||
|
||||
Reference in New Issue
Block a user