feat: estimate the time remaining from bytes and observed rate
Build and publish container / build (pull_request) Successful in 2m13s
Build and publish container / build (pull_request) Successful in 2m13s
rsync reports each file's size with %l as it completes, which is all an estimate needs: bytes done over time elapsed is the same arithmetic rsync would do internally, and requires nothing it does not already print. The scan pass now sums those sizes as well as counting files, so both a percentage and an estimate have a real denominator. The rate is measured over a trailing thirty seconds rather than the whole run, so it follows a device that slows down instead of averaging the slowdown away -- which for a card reader that thermally throttles, or a USB link that renegotiates after an hour, is the difference between a useful estimate and a reassuring one. Below two seconds no rate is reported at all. The first handful of files arrive microseconds apart, and dividing by that window produces a rate in the gigabytes per second and an estimate of zero, which is worse than showing nothing. Directory entries are excluded from the byte total as well as the file count. rsync reports them with a 4096 inode size, which across six thousand album directories is several megabytes of transfer that never happens.
This commit is contained in:
+13
-7
@@ -46,9 +46,10 @@ catches an unmounted device: /media/IPOD/Music then resolves to the host's own
|
||||
root filesystem, and this refuses to empty that.
|
||||
|
||||
Progress is one line that rewrites itself, showing the album currently going
|
||||
across and how far through the transfer is. Working the total out first means a
|
||||
second pass over the tree, which is the price of a percentage that means
|
||||
something; rsync's own is computed against a file list it is still building.
|
||||
across, how far through the transfer is, the rate, and an estimate of what is
|
||||
left. Working the totals out first means a second pass over the tree, which is
|
||||
the price of figures that mean something; rsync's own percentage is computed
|
||||
against a file list it is still building.
|
||||
|
||||
Reach the device with the Apple firmware's disk mode: Menu+Select to reboot,
|
||||
then immediately Select+Play. Power off afterwards by holding Play.
|
||||
@@ -175,12 +176,17 @@ fi
|
||||
# second pass over the tree but means the transfer can show a real percentage
|
||||
# rather than a number that grows as rsync discovers more work.
|
||||
printf 'sync-to-ipod: working out what needs copying...\n' >&2
|
||||
total=$(rsync "${options[@]}" --dry-run --out-format='%n' "$mirror/" "$destination/" |
|
||||
grep -cve '/$' || true)
|
||||
# %l is the file's size, which is what makes an estimate possible. Directories
|
||||
# are dropped: rsync reports those too, with an inode size that would inflate
|
||||
# the total by several megabytes of nothing.
|
||||
counted=$(rsync "${options[@]}" --dry-run --out-format='%l %n' "$mirror/" "$destination/" |
|
||||
awk '!/\/$/ { files++; bytes += $1 } END { print files + 0, bytes + 0 }')
|
||||
total=${counted% *}
|
||||
total_bytes=${counted#* }
|
||||
printf 'sync-to-ipod: %s files to copy\n' "$total" >&2
|
||||
|
||||
rsync "${options[@]}" --out-format='%n' "$mirror/" "$destination/" |
|
||||
python3 "$here/rsync_progress.py" --total "$total"
|
||||
rsync "${options[@]}" --out-format='%l %n' "$mirror/" "$destination/" |
|
||||
python3 "$here/rsync_progress.py" --total "$total" --bytes "$total_bytes"
|
||||
status=${PIPESTATUS[0]}
|
||||
[ "$status" -eq 0 ] || die "rsync exited $status"
|
||||
|
||||
|
||||
Reference in New Issue
Block a user