On Wed, Feb 17, 2010 at 8:56 AM, Edward Ned Harvey <[email protected]>wrote:
> > rsync can handle sparse files by using the -S (--sparse) option. > > > However, it can take longer as it's doing a bunch of processing > > > instead of blindly sending everything over the wire. You are trading > > > off bandwidth use for CPU use. On a local network, this tradeoff may > > > not be worth it. Personally I use rsync on the local network since it > > > also gives me that ability to resume, preserve ownership, etc... > > That’s the problem. When I used rsync to backup the files, the incremental > took over 2x longer than the initial. Why would anybody ever do an > incremental in that case? > > The goal is to do incrementals, and minimize the blocks sent, down to > something in the vicinity of the number of blocks changed. And to do this > in a length of time which is reasonably short, so you’re comfortable doing > it frequently. > rsync *is* minimizing the number of blocks sent, that's why it takes longer -- it needs to figure out which blocks are the ones that changed. But in one sentence you're talking about time, and the next you're talking about minimizing blocks sent (bandwidth use). You need to figure out which one you want. The amount of time it takes is not the only indicator you should be looking at. An rsync copy is going to take significantly less bandwidth than copying the whole file. If bandwidth is not your primary concern (like on a LAN), then this trade-off may not be worth it for you, but don't confuse the "amount of time it takes" with the "amount of bandwidth it uses". > > Have you looked into piping your backups through gzip? Any sparse file > > > would get compressed down to almost nothing, though it also might take > > > a little more time. > > Yes, I’ve done this, and you’re right it does compress all the serial 0’s > down to essentially zero size. However, you’re still sending the whole > file, not just a subset of changed blocks. So it still takes the time of a > full backup, not just an incremental. > Only other thing I can think of is doing a split or something, but you're right that's not the best way either. You might also be able to create a binary diff, but I'm not sure how long that would take.
_______________________________________________ Tech mailing list [email protected] http://lopsa.org/cgi-bin/mailman/listinfo/tech This list provided by the League of Professional System Administrators http://lopsa.org/
