Re: [BackupPC-users] Pool filesystem wierdness
On Tue, 2012-05-29 at 19:27 -0400, Brad Alexander wrote: I know it is bad form to respond to one's own post, but I was digging around in my Munin graphs, and noticed that the filesystem skyrocked from about 70% to 100% late on the 26th or early on the 27th. I have included both the 4-week pool graph from the backup machine and the weekly graph from my munin server. Note that while the pool filesystem is smooth and even at around 300GB, the munin graph climbs sharply...I wasn't able to find anything that might have caused it, though doing df on the pc directory, with all it's hard links, seems to be a Bad Idea. I've been caught out by things like thunderbird indexing 10 gig of mail suddenly and adding a massive chunk to my backups. You can use du on parts of the pool file system as modern versions of du will skip hardlinks after it's counted the first instance of them. -- Tim Fletcher t...@night-shade.org.uk -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Pool filesystem wierdness
On 2012-05-30 00:27, Brad Alexander wrote: I know it is bad form to respond to one's own post, but I was digging around in my Munin graphs, and noticed that the filesystem skyrocked from about 70% to 100% late on the 26th or early on the 27th. I have included both the 4-week pool graph from the backup machine and the weekly graph from my munin server. Note that while the pool filesystem is smooth and even at around 300GB, the munin graph climbs sharply... This indicates the disk usage is outside the pool. Using du on these directories would be bad (take a long time, not be helpful due to hard links): cpool pool pc Everything else you can check. Please do so first. 1. Do you have anything else in this filesystem, or is it mounted on /var/lib/backuppc? 2. Check a few backups made during the window that usage increased. Are they linked into the pool? For instance: root@backup:/var/lib/backuppc/pc/router07/547# stat f%2f/fvar/flog/fsyslog File: `f%2f/fvar/flog/fsyslog' Size: 20440 Blocks: 40 IO Block: 4096 regular file Device: fb00h/64256dInode: 105783860 Links: 2 Access: (0640/-rw-r-) Uid: ( 106/backuppc) Gid: ( 116/backuppc) Access: 2012-05-24 00:08:56.727735552 + Modify: 2012-05-25 02:31:52.655574930 + Change: 2012-05-25 02:31:52.655574930 + This is a file that is unique on every host, but it has two links. Meaning it exists here in the pc directory, and in the pool. If this is 1, this backup is not linked into the pool and is contributing to this usage problem. Regards, Tyler -- A bad analogy is like a leaky screwdriver. -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
[BackupPC-users] restore options
Is there an easy way to set restore options (using rsync), I want to not restore files on the destination that have mod time newer than the backup. -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] restore options
Try RsyncRestoreArgs. GUI - host - Edit Config - Xfer. Regards, Tyler On 2012-05-30 13:51, Neal Becker wrote: Is there an easy way to set restore options (using rsync), I want to not restore files on the destination that have mod time newer than the backup. -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/ -- We should forget about small efficiencies, say about 97% of the time; premature optimization is the root of all evil. -- Donald Knuth -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Pool filesystem wierdness
I think I just found it. Ironically enough, it was my workstation. I have an external drive that is normally plugged into my laptop for files I need to transport I had plugged (and left plugged) this drive into my desktop, which was being indexed and attempted to be backed up. Apparently, this filled the filesystem, and the backup never completed, but the files from the partial backup were apparently stored somewhere. Once I umounted the portable hard drive, the backup completed, and the drive space went back down to normal levels. I have since commented out all of /media. Thanks, --b On Wed, May 30, 2012 at 6:59 AM, Tyler J. Wagner ty...@tolaris.com wrote: On 2012-05-30 00:27, Brad Alexander wrote: I know it is bad form to respond to one's own post, but I was digging around in my Munin graphs, and noticed that the filesystem skyrocked from about 70% to 100% late on the 26th or early on the 27th. I have included both the 4-week pool graph from the backup machine and the weekly graph from my munin server. Note that while the pool filesystem is smooth and even at around 300GB, the munin graph climbs sharply... This indicates the disk usage is outside the pool. Using du on these directories would be bad (take a long time, not be helpful due to hard links): cpool pool pc Everything else you can check. Please do so first. 1. Do you have anything else in this filesystem, or is it mounted on /var/lib/backuppc? 2. Check a few backups made during the window that usage increased. Are they linked into the pool? For instance: root@backup:/var/lib/backuppc/pc/router07/547# stat f%2f/fvar/flog/fsyslog File: `f%2f/fvar/flog/fsyslog' Size: 20440 Blocks: 40 IO Block: 4096 regular file Device: fb00h/64256d Inode: 105783860 Links: 2 Access: (0640/-rw-r-) Uid: ( 106/backuppc) Gid: ( 116/backuppc) Access: 2012-05-24 00:08:56.727735552 + Modify: 2012-05-25 02:31:52.655574930 + Change: 2012-05-25 02:31:52.655574930 + This is a file that is unique on every host, but it has two links. Meaning it exists here in the pc directory, and in the pool. If this is 1, this backup is not linked into the pool and is contributing to this usage problem. Regards, Tyler -- A bad analogy is like a leaky screwdriver. -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
[BackupPC-users] backup times out (signal ALRM) when using --acls and --xattrs rsync options
BackupPC Server:
---
RHEL6.2
BackupPC-3.2.1-7.el6.x86_64 (EPEL)
rsync-3.0.6-5.el6_0.1.x86_64
/var/lib/BackupPC is an NFS mount point (i.e. the BackupPC server is
an NFS client)
mounted via /etc/fstab entry:
nfsserver.example.comoh :/mnt/backup/gs-444-e10285
/var/lib/BackupPCnfs exec,suid,rw,rsize=8192,wsize=8192 1 1
rsync --version
rsync version 3.0.6 protocol version 30
Copyright (C) 1996-2009 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
append, ACLs, xattrs, iconv, symtimes
NFS server:
---
RHEL4.9
rsync-3.0.7-1.el4.rf (from dag repo):
rsync --version
rsync version 3.0.7 protocol version 30
Copyright (C) 1996-2009 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
64-bit files, 64-bit inums, 32-bit timestamps, 64-bit long ints,
socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
append, ACLs, xattrs, iconv, no symtimes
I'm trying to backup localhost. The first backup I ran was successful.
This was my localhost.pl config:
$Conf{RsyncClientCmd} = '/usr/bin/sudo $rsyncPath $argList+';
$Conf{RsyncClientRestoreCmd} = '/usr/bin/sudo $rsyncPath $argList+';
$Conf{RsyncArgs} = [
'--numeric-ids',
'--perms',
'--owner',
'--group',
'-D',
'--links',
'--hard-links',
'--times',
'--block-size=2048',
'--recursive',
'--exclude',
'/var/lib/BackupPC'
];
Of course I have setup sudo for backuppc to run rsync with no password.
Then I decided that I wanted to make sure the metadata was backed up, so
after a little googling it seems that adding --acls and --xattrs to the
rsync options would do it:
$Conf{RsyncArgs} = [
'--numeric-ids',
'--perms',
'--owner',
'--group',
'-D',
'--links',
'--hard-links',
'--times',
'--block-size=2048',
'--recursive',
'--acls',
'--xattrs',
'--exclude',
'/var/lib/BackupPC'
];
However, now when a backup runs, it times out after 20 hours with signal
ALRM; e.g:
2012-05-19 11:00:12 incr backup started back to 2012-05-16 14:50:08 (backup #0)
for directory /
2012-05-20 07:00:12 Aborting backup up after signal ALRM
2012-05-20 07:00:13 Got fatal error during xfer (aborted by signal=ALRM)
I can see that when the backup is running that $TopDir/pc/localhost/new
is not growing -- it stays at 8K during the entire 20 hours.
I took out the two new options and ran another incr backup, it seems to
be working fine (it's running now, and $TopDir/pc/localhost/new is
growing in size)
Suggestions, please?
--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Robert Jacobson robert.c.jacob...@nasa.gov
Lead System Admin Solar Dynamics Observatory (SDO)
Bldg 14, E222 (301) 286-1591
--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] BackupPC Windows Client
First, thanks for doing this. I've had the original post bookmarked for quite some time (years, it seems) with plans to implement, and this simplifies the process quite a bit. Performance is much better than rsync-over-autofs-smb (for any number of reasons). 5) I got a passing error message from the postusercmd.sh at one stage, and would suggest the following change: if [ ! -z $xferOK ] [ $xferOK-eq 1 ]; then Basically, if xferOK is not set, then [ $xferOK-eq 1 ] generates an error (and does not match), with my change it will not generate an error, and will still not match. I ran into that error postusercmd.sh: line 10: [: -eq: unary operator expected as well - even on successful runs. It looks like $xferOK is never declared, but is just passed in. postusercmd.sh BOX=$1 *XFEROK=$2* $WINEXE -U $UNAME -W $WRKGRP --password=$PWD //$BOX 'cmd /c echo '1' c:\backuppc\wake.up' - echo Rsync and shadow copy unloaded if [ *$XFEROK* -eq 1 ]; then if [ -f $WOLDIR/$BOX.state ]; then read wasoff $WOLDIR/$BOX.state echo -n State before backup: Thanks for all your work on this! -Kris Lou Not only are you exactly correct, but your fix is exactly what I left out when I posted it the first time (I apparently deleted that line and hadn't noticed.) The client side hasn't changed, but I'm updating the scripts with the fixes this week, and will have it posted as soon as I do. Thanks! -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] BackupPC Windows Client
On 31/05/12 00:59, Michael Stowe wrote: Not only are you exactly correct, but your fix is exactly what I left out when I posted it the first time (I apparently deleted that line and hadn't noticed.) The client side hasn't changed, but I'm updating the scripts with the fixes this week, and will have it posted as soon as I do. Just a little reminder that on each machine I've tried to install the client on, the rsyncd.secrets file wasn't created. Also, when using a username other than Administrator (for rsync) the install also fails to use the correct username in the rsyncd.conf file. Haven't had a chance to install on a 2008 server yet, but hope to have more of a play in the coming weeks. Regards, Adam -- Adam Goryachev Website Managers www.websitemanagers.com.au -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Pool filesystem wierdness
On Wed, 2012-05-30 at 10:15 -0400, Brad Alexander wrote: I think I just found it. Ironically enough, it was my workstation. I have an external drive that is normally plugged into my laptop for files I need to transport I had plugged (and left plugged) this drive into my desktop, which was being indexed and attempted to be backed up. Apparently, this filled the filesystem, and the backup never completed, but the files from the partial backup were apparently stored somewhere. Once I umounted the portable hard drive, the backup completed, and the drive space went back down to normal levels. I have since commented out all of /media. Changing the rsync flags to include --one-file-system at the top level will stop this catching you out again, I think it should be a configuration default. -- Tim Fletcher t...@night-shade.org.uk -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
[BackupPC-users] cheap encryption
Using rsync (via ssh), default is aes encryption, which is expensive.
I wanted to try setting
Host * Ciphers arcfour,blowfish-cbc
I put that in user backuppc .ssh/config, but that didn't seem to work
(according
to the output of ps, not showing the args to ssh).
I did find that putting it in
$Conf{RsyncArgs}
as
'-e ssh -c arcfour,blowfish-cbc'
(note the quotes)
appears to be working (I can see the args in ps, and the backup is running).
--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] cheap encryption
On Wed, May 30, 2012 at 1:05 PM, Neal Becker ndbeck...@gmail.com wrote:
Using rsync (via ssh), default is aes encryption, which is expensive.
I wanted to try setting
Host * Ciphers arcfour,blowfish-cbc
I put that in user backuppc .ssh/config, but that didn't seem to work
(according
to the output of ps, not showing the args to ssh).
I did find that putting it in
$Conf{RsyncArgs}
as
'-e ssh -c arcfour,blowfish-cbc'
(note the quotes)
appears to be working (I can see the args in ps, and the backup is running).
Sss is very pick about permissions. Make sure the config file is
owned by backuppc and not accessible by others. If you don't mind
changing the default for other users on the server you can make the
change in /etc/ssh/ssh_config.
--
Les Mikesell
lesmikes...@gmail.com
--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Pool filesystem wierdness
I can't really do that, since I tend to segregate my filesystems anyway. I used to have my non-standard stuff in /media/archive (which was backed up), I just turned off /media/cdrom and an nfs mount there. When I rebuilt, I decided /data was a better choice, but didn't think to turn off /media. --b On Wed, May 30, 2012 at 12:12 PM, Tim Fletcher t...@night-shade.org.uk wrote: On Wed, 2012-05-30 at 10:15 -0400, Brad Alexander wrote: I think I just found it. Ironically enough, it was my workstation. I have an external drive that is normally plugged into my laptop for files I need to transport I had plugged (and left plugged) this drive into my desktop, which was being indexed and attempted to be backed up. Apparently, this filled the filesystem, and the backup never completed, but the files from the partial backup were apparently stored somewhere. Once I umounted the portable hard drive, the backup completed, and the drive space went back down to normal levels. I have since commented out all of /media. Changing the rsync flags to include --one-file-system at the top level will stop this catching you out again, I think it should be a configuration default. -- Tim Fletcher t...@night-shade.org.uk -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/ -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Pool filesystem wierdness
On Wed, May 30, 2012 at 8:00 PM, Brad Alexander stor...@gmail.com wrote: I can't really do that, since I tend to segregate my filesystems anyway. I used to have my non-standard stuff in /media/archive (which was backed up), I just turned off /media/cdrom and an nfs mount there. When I rebuilt, I decided /data was a better choice, but didn't think to turn off /media. All you have to do is add explicit 'shares' for every mount point that you do want to back up. As a bonus, --one-file-system will automatically exclude the pseudo-mounts like /proc and /sys, etc. -- Les Mikesell lesmikes...@gmail.com -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
