Hi,
Les Mikesell wrote on 27.03.2007 at 01:03:32 [Re: [BackupPC-users] RSync v.
Tar]:
> Jesse Proudman wrote:
> > I've got one customer who's server has taken 3600 minutes to
> > backup. 77 Gigs of Data. 1,972,859 small files. Would tar be
> > better or make this faster? It's directly connected via 100 Mbit to
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > the backup box.
^^^^^^^^^^^^^^
> If the files don't change frequently, tar incremental runs will be much
> faster because they are based only on the target timestamps while rsync
> will load the entire directory at both ends and compare them.
if you ask me, regardless of how your data changes, tar is the way to go,
not rsync, especially for *full* backups. With a direct 100 MBit connection,
there's not much point in spending (lots of) CPU time for saving bandwidth
- not with 2 million files. rsync is good for low bandwidth connections,
where the link severely limits the transfer and speeding it up makes a real
difference. In your case, your link speed is in the same order of magnitude
as your disk I/O performance (considering our favorite topic, the "seek times"
on the pool file system, the network link may in fact not even be the limiting
factor - it clearly isn't, as 77 GB would take slightly more than 2 hours to
transfer over a 100 MBit link, and rsync is not making it go faster than
that ;-).
rsync has additional benefits concerning finding and backing up new (or
moved) files with old timestamps and deleted files on *incremental* backups,
but keeping the list of 2 million files in memory will probably be a problem,
as it possibly was with your full (?) backup (how much memory do the BackupPC
server and the backed up host have?).
(Les: if the files *do* change frequently, there's even less speedup to get
from using rsync. Only frequent metadata changes without file content
changes would give rsync an advantage - assuming it is faster to figure out
that the file is identical than to simply send it over the network.)
Regards,
Holger
P.S.: rsync checksum caching *might* make a difference starting from the
third backup, but I read that it's less improvement than one might
expect.
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
BackupPC-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/
