Hi,
Bowie Bailey wrote on 2018-06-14 15:38:44 -0400 [Re: [BackupPC-users] Slow
local backup]:
> On 6/14/2018 3:27 PM, Michael Stowe wrote:
> > On 2018-06-14 10:05, Bowie Bailey wrote:
> >
> > I just installed BackupPC v4 on a CentOS 7 server with 4G ram. I
> > am trying to back up a local 318G filesystem. I am using rsyncd
> > over the loopback connection. It has been running for 17 hours so
> > far and has backed up less than half of the directory (based on
> > the size of the backup filesystem). Running top does not show any
> > excessive cpu or iowait. ???free??? shows no swap usage and 1.5G
> > available memory.
if I'm not miscalculating, that is roughly 10GB/hour or 3MB/s. From what I've
read about BackupPCv3 performance, that wouldn't seem extremely unreasonable,
especially for a first backup. For V4, I have no idea what the common figures
are. The C implementation (rsync_bpc) might have a performance benefit. But I
would expect one core of your CPU to be almost 100% busy, while the others may
be idle. Depending on which figure you are looking at, that might not seem
excessive, but it's the most you can get for single threaded compression
performance (you *are* using compression, right?).
> [...]
> I use rsyncd rather than rsync to avoid the ssh overhead.
For a local backup, you can achieve the same by using 'sudo' instead of 'ssh'.
At least you could with V3. I've forgotten if and how you can use 'sudo' with
rsync in V4.
> I expected a backup done via the loopback interface to be fast since it
> doesn't have the normal networking bandwidth limitations.
Well, yes, but you still have a network stack and probably at least two copies
between kernel and user space. loopback networking does not come for free. A
quick 'netperf' test gives me a bit less than 9 Gbit/s throughput. That's
certainly faster than Gbit networking, but only by a factor of 10.
More importantly, the other limitations don't change - disk and compression
speed, for instance. If your bottleneck is not the network, a faster network
won't change anything. Keep in mind that a local backup means that client
and server are the same computer, so a loopback backup might actually be
*slower* than a remote backup. Your source file system and BackupPC pool are
on different physical disks, hopefully?
> > Is it normal for the backup to take this long?
> >
> > While that's hard to guess without knowing the particulars of your
> > system, I'm going to go out on a limb and say, no. No it is not.
I believe it is still important whether it is the first backup or not. The
recommendation used to be "ignore the timing of the first backup - fix your
problem only if the second (or third) backup is still too slow". That would
still seem to apply to BackupPCv4.
> > Is there a better way to back up a local filesystem?
> >
> > Personally, I use rsync (not rsyncd) and in the cases where I have
> > experienced slowness, it was due to poorly chosen rsync parameters (I
> > note that this would not differ between rsync and rsyncd), a broken
> > filesystem, or a specific bug in a specific version of rsync.
>
> How do you go about setting up rsync so that it does a local copy rather
> than going through ssh over the network?
I would try setting $Conf {RsyncSshArgs} = [ '-e', '/usr/bin/sudo' ], but
you might run into quoting problems. You could then try using a script
containing something like 'exec /usr/bin/sudo $@'. Or see if using 'ssh'
makes much of a difference at all ...
Hope that helps.
Regards,
Holger
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
