It's great you brought up bbcp. I didn't factor this into my initial email, but if we could further split up large files into N chunks and transport them concurrently, that would provide massive benefits as well.
It's clear there are multiple areas for improvement that would let us do away with having to use other tools for various scenarios. If we could put some planning/development time in even the easiest-to-implement cases, it would be highly-beneficial to a large number of users. -AmeirFrom: [email protected]<[email protected]> To: <[email protected]> Sent: Sunday, September 8, 2013 Subject: Re: Native Parallelization in rsync On 08/09/13 18:28, [email protected] wrote: > There is another rather common case I think where this would help: > when the destination directory is located on a NAS over NFS. I think the biggest use-case is high bandwidth, high latency links. eg you have a 50Mbs WAN link to a site in another country, and can never get more than (say) 5Mbs. Parallelizing the data transfer could easily push that up to 20-30Mbs ...and there is a "competitor" to rsync that does this - bbcp. Mirror a directory from hostA to hostB using 'N' tcp streams. Runs like the clappers :-) -- Cheers Jason Haar Information Security Manager, Trimble Navigation Ltd. Phone: +1 408 481 8171 PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1 -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
-- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
