Tony Schreiner wrote: > On Jul 3, 2008, at 4:07 PM, Evren Yurtesen wrote: > >> Renke Brausse wrote: >>> Hello Tony, >>>> I've written before about backups involving very big files that >>>> seem to execute slowly. >>>> >>>> What can be slowing things down so much? Except for this >>>> operation, everything else runs about as I would expect. >>> I have no clue what the reason is but I experienced that backups of >>> large files are much faster with tar over ssh instead of rsync >>> over ssh. >>> Not an explanation but maybe this can solve your problem. >> I believe the reason for this is how rsync works. It normally tries >> to transfer only the changed parts of the file. This is to save >> bandwidth, to do this, it has to scan the whole file on both sides >> (I guess). This is unnecessary unless you are over slow links. You >> might want to try the whole-file option with rsync: >> >> -W, --whole-file copy files whole (w/o delta- >> xfer algorithm) >> >> Please let us know the results, as a side-note if you still want to >> shrink the transferred file size you can use the ssh compression >> with -C option of ssh. >> >> Thanks, >> Evren > > > Reporting back on this. Using the -W option did not make much > difference. The dumps in question continue to run for a long time and > eventually fail with an ALARM. > > For the time being I am excluding the really large files in question. > I may try tar instead of rsync at some point as was suggested. > > Tony Schreiner
Are you sure that -W option was active for sure? Did you check with ps axwww to be sure? or just took it granted that it was active? Just asking because I have made several times similar mistake myself :) By the way, if you are using compression, also enabling --checksum-seed=32761 will make a dramatic speed difference. (although manual suggests that this can be visible only on 3rd full backup). Thanks, Evren ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08 _______________________________________________ BackupPC-users mailing list [email protected] List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
