-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Depends on what block level tool you use. Which will often depend on what kind of block device, filesystem, and operating system is involved.
On 09/13/11 02:14, David Somers-Harris wrote: > A block level tool won't do a differential back up like rsync does will it? > > On Tue, Sep 13, 2011 at 11:04, Kevin Korb <[email protected] > <mailto:[email protected]>> wrote: > > The rsync is doing a whole lot more than transferring 600MB of data. It > is transferring 600MB of differences. It is going through 27.4 million > files on one end and possibly that many on the other. That is a 402MB > listing of files plus as many as 54.8 million calls to stat(). > > Rsync is not about minimizing transfer time. It is about minimizing > network IO at the expense of CPU and disk IO. Rsync reduced your > network IO from 185.39GB to 600MB only 200 of which was actually > differences in your data. > > IOW rsync did its job. > > The unfortunate truth is that no file based tool will perform anywhere > near wire speeds when it has millions of files to deal with. If you > want something wire speed then you should look for a block level tool. > > On 09/12/11 21:54, David Somers-Harris wrote: >> No, I don't have checksum based checking enabled. > >> Sorry for neglecting to mention the options I'm using, here they are: > >> *rsync -vah --exclude=".*" --delete --stats* > >> Would anything in there cause it to be unnecessarily slow? It's > taking 3 >> hours but shouldn't take more than 5 minutes... > > >> On Tue, Sep 13, 2011 at 00:02, Sandon Van Ness > <[email protected] <mailto:[email protected]> >> <mailto:[email protected] <mailto:[email protected]>>> wrote: > >> It could definitely be normal if say you have checksum based >> checking enabled... > > >> On 09/12/2011 01:56 AM, David Somers-Harris wrote: > > >> Is it normal for rsync to take 3 hours on this transfer? > >> Number of files: 27419348 >> Number of files transferred: 19501 >> Total file size: 185.39G bytes >> Total transferred file size: 195.92M bytes >> Literal data: 195.68M bytes >> Matched data: 241.09K bytes >> File list size: 402.01M >> File list generation time: 0.561 seconds >> File list transfer time: 0.000 seconds >> Total bytes sent: 600.61M >> Total bytes received: 2.44M > >> sent 600.61M bytes received 2.44M bytes 54.42K bytes/sec >> total size is 185.39G speedup is 307.42 > > >> A normal transfer of 600 MB for me between these two servers >> only takes a few seconds.... Is there some way I can speed > this up? > > > > - -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html <http://www.catb.org/%7Eesr/faqs/smart-questions.html> - -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ Kevin Korb Phone: (407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. [email protected] (work) Orlando, Florida [email protected] (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk5vSjcACgkQVKC1jlbQAQfy8QCg2KipjXb6vMGc4F090eF12RRC FGIAnA2BHTdw70o1M61kRGJovqi30plw =tppS -----END PGP SIGNATURE----- -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
