Is this really your first rdiff-backup to this location? If you have any previous rdiff-backup runs to this repository then the situation is complicated by rdiff-backup's need to create a new set of reverse diff files to be able to regress to previous file contents.
What is your /tmp location? rdiff-backup uses this location for some operations though not AFAIK for standard backup runs. Still, if /tmp is on encfs maybe it could be a culprit; you can override rdiff-backup's temporary file location with --tempdir and --remote-tempdir. Might also be worth trying --ssh-no-compression. Dominic http://www.timedicer.co.uk On 28 March 2016 at 14:37, Derek Atkins <de...@ihtfp.com> wrote: > Just a quick update: I tried making these changes on both sides and it > really didn't make a difference. Full backup of 202852072 Kbytes > required 2267m25.913s (previously it took 2346m57.800s, so it only sped > up a factor of 3%. > > Only thing I have not yet tried is running a raw rsync to see how fast > that runs. I'll do that next. > > So, back to my orignal question: anyone have any idea how to get initial > transfers to run faster (or indeed any significant data transfers)? > > Thanks, > > -derek > > "Derek Atkins" <de...@ihtfp.com> writes: > > > Hi, > > > > I'm trying to use rdiff-backup to backup a bunch of servers. One > > particular server contains about 160GB of data, but when I try to perform > > the rdiff-backup it's saving the data at a measly 1MB/s. > > > > Here's my configuration: > > > > [server] <--ssh--> [backup-server]{encfs} <--nfs--> [freenas] > > > > I ran a bunch of tests to try to figure out my bottlenecks. > > > > I ran a bunch of dd tests (using dd if=/dev/zero bs=1M count=1000) on the > > backup server. Going directly to FreeNAS via NFS (bybassing encfs) I get > > 50.2MB/s. If I run dd directly on the backup server (through encfs) I > get > > 20.1MB/s. If I go over SSH from the backup server to the target server > > and run the dd on the target server, then write to FreeNas through encfs > > declines to 7.6MB/s. > > > > Note that in those SSH tests, however, I forgot to turn off compression. > > When I do that, the throughput for the dd test reduced to 6.6BM/s. (Note > > that this is running simultaneously with a running rdiff-backup, so it's > > possible that they are reducing performance). > > > > Then I ran an scp test to the same target server; copying about 1.4GB of > > photos. Files ranged in size from 10KB to 5MB. When run in standard > mode > > (displaying each file status) I got 4.4MB/s. Running in quiet mode I get > > 5.1MB/s. > > > > So clearly the bottleneck is in rdiff-backup -- performance (IMHO) should > > not be significantly slower than the last dd-over-ssh test. It appears > > rdiff-backup is slowing me down by a factor of 5x throughput versus scp. > > > > I found a message from Ben from 2005 where he suggests increasing the > > blocksize and conn_bufsiz settings in Globals.py: > > > https://lists.gnu.org/archive/html/rdiff-backup-users/2005-10/msg00062.html > > > > What he didn't say was whether this needed to be changed on the target > > server, the backup server, or both. Nor do I know if that would actually > > help this situation. > > > > Do you have any ideas? > > > > Thanks, > > > > -derek > > > > PS: According to rpm, both systems are running version 1.2.8. > > -- > Derek Atkins 617-623-3745 > de...@ihtfp.com www.ihtfp.com > Computer and Internet Security Consultant > > _______________________________________________ > rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org > https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users > Wiki URL: > http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki >
_______________________________________________ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
