Well what if it was FreeBSD on ext3 on a hard disk? What block level tools are available for that setup which can do differential back ups?
On Tue, Sep 13, 2011 at 21:19, Kevin Korb <[email protected]> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Depends on what block level tool you use. Which will often depend on > what kind of block device, filesystem, and operating system is involved. > > On 09/13/11 02:14, David Somers-Harris wrote: > > A block level tool won't do a differential back up like rsync does will > it? > > > > On Tue, Sep 13, 2011 at 11:04, Kevin Korb <[email protected] > > <mailto:[email protected]>> wrote: > > > > The rsync is doing a whole lot more than transferring 600MB of data. It > > is transferring 600MB of differences. It is going through 27.4 million > > files on one end and possibly that many on the other. That is a 402MB > > listing of files plus as many as 54.8 million calls to stat(). > > > > Rsync is not about minimizing transfer time. It is about minimizing > > network IO at the expense of CPU and disk IO. Rsync reduced your > > network IO from 185.39GB to 600MB only 200 of which was actually > > differences in your data. > > > > IOW rsync did its job. > > > > The unfortunate truth is that no file based tool will perform anywhere > > near wire speeds when it has millions of files to deal with. If you > > want something wire speed then you should look for a block level tool. > > > > On 09/12/11 21:54, David Somers-Harris wrote: > >> No, I don't have checksum based checking enabled. > > > >> Sorry for neglecting to mention the options I'm using, here they are: > > > >> *rsync -vah --exclude=".*" --delete --stats* > > > >> Would anything in there cause it to be unnecessarily slow? It's > > taking 3 > >> hours but shouldn't take more than 5 minutes... > > > > > >> On Tue, Sep 13, 2011 at 00:02, Sandon Van Ness > > <[email protected] <mailto:[email protected]> > >> <mailto:[email protected] <mailto:[email protected]>>> wrote: > > > >> It could definitely be normal if say you have checksum based > >> checking enabled... > > > > > >> On 09/12/2011 01:56 AM, David Somers-Harris wrote: > > > > > >> Is it normal for rsync to take 3 hours on this transfer? > > > >> Number of files: 27419348 > >> Number of files transferred: 19501 > >> Total file size: 185.39G bytes > >> Total transferred file size: 195.92M bytes > >> Literal data: 195.68M bytes > >> Matched data: 241.09K bytes > >> File list size: 402.01M > >> File list generation time: 0.561 seconds > >> File list transfer time: 0.000 seconds > >> Total bytes sent: 600.61M > >> Total bytes received: 2.44M > > > >> sent 600.61M bytes received 2.44M bytes 54.42K bytes/sec > >> total size is 185.39G speedup is 307.42 > > > > > >> A normal transfer of 600 MB for me between these two servers > >> only takes a few seconds.... Is there some way I can speed > > this up? > > > > > > > > > - -- > Please use reply-all for most replies to avoid omitting the mailing > list. > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: > http://www.catb.org/~esr/faqs/smart-questions.html > <http://www.catb.org/%7Eesr/faqs/smart-questions.html> > > - -- > ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ > Kevin Korb Phone: (407) 252-6853 > Systems Administrator Internet: > FutureQuest, Inc. [email protected] (work) > Orlando, Florida [email protected] (personal) > Web page: http://www.sanitarium.net/ > PGP public key available on web site. > ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.17 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iEYEARECAAYFAk5vSjcACgkQVKC1jlbQAQfy8QCg2KipjXb6vMGc4F090eF12RRC > FGIAnA2BHTdw70o1M61kRGJovqi30plw > =tppS > -----END PGP SIGNATURE----- > -- > Please use reply-all for most replies to avoid omitting the mailing list. > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html >
-- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
