https://bugzilla.samba.org/show_bug.cgi?id=13645
Bug ID: 13645 Summary: Improve efficiency when resuming transfer of large files Product: rsync Version: 3.0.9 Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 Component: core Assignee: way...@samba.org Reporter: pe1...@amsat.org QA Contact: rsync...@samba.org When transferring large files over a slow network, we interrupt rsync at the beginning of business hours leaving the transfer unfinished. The command used is: rsync -av --inplace --bwlimit=400 hostname::module /dest When restarting the transfer, a lot of time is "wasted" while first the local system is reading the partially transferred file and sends the checksums to the remote, which only then starts to read the source file until it finds something to transfer. So nothing happens until 2 times the time required to read the partial transfer from the disks! When the partial file is many many GB, this can take hours. Suggestions: 1. when the source is larger than the destination, immediately begin to transfer from the offset in the source equal to the size of the destination. it is already known that this part will have to be transferred. 2. try to do the reading of the partial file at the destination and the same part of the source in parallel (so the time is halved), and preferably also in parallel to 1. Of course these optimizations (at least #2) may actually decrease performance when the transfer is local (not over slow network) and the disk read rate is negatively affected by reading at two different places in parallel. So #2 should only be attempted when the transfer is over a network. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html