feat: estimate the time remaining from bytes and observed rate
Build and publish container / build (pull_request) Successful in 2m13s

rsync reports each file's size with %l as it completes, which is all an
estimate needs: bytes done over time elapsed is the same arithmetic rsync would
do internally, and requires nothing it does not already print. The scan pass now
sums those sizes as well as counting files, so both a percentage and an estimate
have a real denominator.

The rate is measured over a trailing thirty seconds rather than the whole run,
so it follows a device that slows down instead of averaging the slowdown away --
which for a card reader that thermally throttles, or a USB link that renegotiates
after an hour, is the difference between a useful estimate and a reassuring one.

Below two seconds no rate is reported at all. The first handful of files arrive
microseconds apart, and dividing by that window produces a rate in the gigabytes
per second and an estimate of zero, which is worse than showing nothing.

Directory entries are excluded from the byte total as well as the file count.
rsync reports them with a 4096 inode size, which across six thousand album
directories is several megabytes of transfer that never happens.
This commit is contained in:
Emma Thorpe
2026-08-25 12:40:32 +01:00
parent 37b841f009
commit 131c80f5de
4 changed files with 219 additions and 27 deletions
+13 -7
View File
@@ -46,9 +46,10 @@ catches an unmounted device: /media/IPOD/Music then resolves to the host's own
root filesystem, and this refuses to empty that.
Progress is one line that rewrites itself, showing the album currently going
across and how far through the transfer is. Working the total out first means a
second pass over the tree, which is the price of a percentage that means
something; rsync's own is computed against a file list it is still building.
across, how far through the transfer is, the rate, and an estimate of what is
left. Working the totals out first means a second pass over the tree, which is
the price of figures that mean something; rsync's own percentage is computed
against a file list it is still building.
Reach the device with the Apple firmware's disk mode: Menu+Select to reboot,
then immediately Select+Play. Power off afterwards by holding Play.
@@ -175,12 +176,17 @@ fi
# second pass over the tree but means the transfer can show a real percentage
# rather than a number that grows as rsync discovers more work.
printf 'sync-to-ipod: working out what needs copying...\n' >&2
total=$(rsync "${options[@]}" --dry-run --out-format='%n' "$mirror/" "$destination/" |
grep -cve '/$' || true)
# %l is the file's size, which is what makes an estimate possible. Directories
# are dropped: rsync reports those too, with an inode size that would inflate
# the total by several megabytes of nothing.
counted=$(rsync "${options[@]}" --dry-run --out-format='%l %n' "$mirror/" "$destination/" |
awk '!/\/$/ { files++; bytes += $1 } END { print files + 0, bytes + 0 }')
total=${counted% *}
total_bytes=${counted#* }
printf 'sync-to-ipod: %s files to copy\n' "$total" >&2
rsync "${options[@]}" --out-format='%n' "$mirror/" "$destination/" |
python3 "$here/rsync_progress.py" --total "$total"
rsync "${options[@]}" --out-format='%l %n' "$mirror/" "$destination/" |
python3 "$here/rsync_progress.py" --total "$total" --bytes "$total_bytes"
status=${PIPESTATUS[0]}
[ "$status" -eq 0 ] || die "rsync exited $status"