On Wed, March 30, 2016 12:47 pm, Greg Troxel wrote:
>
> "Derek Atkins" <de...@ihtfp.com> writes:
>
>>> I would up the send/receive socket buffers (because it's easy, not
>>> because I think that's the problem), and watch disk/cpu on both sides,
>>> and also run netstat to see if data is piling up in the transmit socket
>>> buffer.
>>
>> Do you mean within rdiff-backup, or at some other level?
>
> On NetBSD I mean bumping up these sysctls
>
> net.inet.tcp.sendspace = 131072
> net.inet.tcp.recvspace = 131072
> net.inet6.tcp6.sendspace = 131072
> net.inet6.tcp6.recvspace = 131072
>
> and presumably that's similar on other systems.
I'll look for the Linux equivalent, however...
> But, if your socket buffers aren't full, that's probably not your
> problem.
... according to repeated netstat the socket buffers are GENERALLY 0,0. I
did see it get up to about 1000, at one point... OH, I take that back,
the SEND Queue value was just up to 243816 on the target system....
>>> FWIW, I used to use rdiff-backup but found it to be nonrobust on
>>> machines with limited (only a few GB) RAM and hundreds of GB of backup.
>>> I have switched to bup.
>>
>> Unfortunately bup is not available on all my target platforms.
>
> bup is python with a little C, and thus seems pretty portable. Where
> isn't it working?
Didn't say it wasn't working. I said "it is not available", meaning there
is no prepackaged RPM for some of my systems. I could certainly try to
piece it together, but of course I prefer to use distro-supplied software
wherever possible. It makes upgrading much easier.
> There is also attic and borg which are similar to bup.
>
>> Maybe I should consider amanda or bacula?
>
> amanda is basically wrappers around dump and tar. If you have 50
> machines and want to do level 0/1/2 to tape and take tapes offsite, it
> works great, after you pay for the LTO tape drive.
I don't have 50 machines. I do have ~10-15. Pretty much all Linux boxes.
Not using tape; backups are all on spinning media.
> Two things to think about:
>
> do you care about deduplication? bup does not only per-file but
> within-file deduplication, so if multiple boxes have the same data it
> doesn't take up extra space
No. At least, not at the block level. I would care about file-level
dedup, but honestly I only care about that from within one server. If I
have multiple systems that happen to have the exact same config file I
don't particularly care about dedupping that. I just don't want a daily
copy of /etc/passwd :)
> Do you really need to back up all platforms, or could you sync from
> some (Android?) to a machine with more disk and back that up? I have
> been using syncthing, which seems to be pretty solid, for syncing
> among Android and regular computers (BSD and OS X). (It is written in
> go so it's not in practice that portable.)
Pretty much all the systems I want to backup are Linux, but different
vintages and such.
Android is different and I back that up differently. I'm just trying to
maximize performance. I've got a GigE network and relatively plenty of
encryption performance; I'd like to leverage that in my backup (and
restore) operations.
Thanks,
-derek
--
Derek Atkins 617-623-3745
de...@ihtfp.com www.ihtfp.com
Computer and Internet Security Consultant
_______________________________________________
rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
