[BackupPC-users] jLib.pm question

2010-12-06 Thread Robin Lee Powell
I've locally modified BackupPC_fixLinks to check files even if they have more than one link. I'm getting a lot of errors like this: Can't read size of /backups/pc/foo--tm50-e00145--tm50-s00339---shared/47/f%2f/fshared/flogs/fapplication.log-20100830.gz from attrib file so calculating

Re: [BackupPC-users] Bizarre form of cpool corruption.

2010-12-06 Thread Robin Lee Powell
On Mon, Dec 06, 2010 at 10:37:52AM -0800, Robin Lee Powell wrote: So I'm writing a script to transfer a client from one host to another, using tarPCCopy, and I'm getting messages like this: Can't find foo--tm50-e00145--tm50-s00339---shared/47/f%2f/fshared/ffoo/fpurchase_order_assets

[BackupPC-users] jLib bug with md5sum stuff

2010-12-06 Thread Robin Lee Powell
substr outside of string at /usr/local/lib64/BackupPC/jLib.pm line 162. Use of uninitialized value in concatenation (.) or string at /usr/local/lib64/BackupPC/jLib.pm line 162. The line(s) in question: $datalast = substr($data[($i-1)%2], $rsize, _128KB-$rsize)

Re: [BackupPC-users] jLib.pm question

2010-12-06 Thread Robin Lee Powell
On Mon, Dec 06, 2010 at 03:33:52PM -0500, Jeffrey J. Kosowsky wrote: Robin Lee Powell wrote at about 11:28:56 -0800 on Monday, December 6, 2010: I've locally modified BackupPC_fixLinks to check files even if they have more than one link. I'm not sure what you mean... the program

Re: [BackupPC-users] Bizarre form of cpool corruption.

2010-12-06 Thread Robin Lee Powell
On Mon, Dec 06, 2010 at 03:48:04PM -0500, Jeffrey J. Kosowsky wrote: Robin Lee Powell wrote at about 10:37:52 -0800 on Monday, December 6, 2010: So I'm writing a script to transfer a client from one host to another, using tarPCCopy, and I'm getting messages like this: Can't

Re: [BackupPC-users] Bizarre form of cpool corruption.

2010-12-06 Thread Robin Lee Powell
On Mon, Dec 06, 2010 at 03:48:04PM -0500, Jeffrey J. Kosowsky wrote: Robin Lee Powell wrote at about 10:37:52 -0800 on Monday, December 6, 2010: So I'm writing a script to transfer a client from one host to another, using tarPCCopy, and I'm getting messages like this: Can't

[BackupPC-users] Another jLib/fixLinks issue.

2010-12-06 Thread Robin Lee Powell
This is *fascinating*. From the actually-fixing-stuff part of the run, I get: ERROR: tm50-s00292__nfs/68/f%2f/fshared/fthepoint/fsite_images/f0042/f4097/fMULTI_medium.jpg - Too many links if added to 59c43b51dbdd9031ba54971e359cdcec to which I say lolwut? and investigate. $ ls -li

Re: [BackupPC-users] Another jLib/fixLinks issue.

2010-12-06 Thread Robin Lee Powell
On Mon, Dec 06, 2010 at 03:40:04PM -0800, Robin Lee Powell wrote: This is *fascinating*. From the actually-fixing-stuff part of the run, I get: ERROR: tm50-s00292__nfs/68/f%2f/fshared/fthepoint/fsite_images/f0042/f4097/fMULTI_medium.jpg - Too many links if added

Re: [BackupPC-users] Bizarre form of cpool corruption.

2010-12-06 Thread Robin Lee Powell
On Mon, Dec 06, 2010 at 01:17:43PM -0800, Robin Lee Powell wrote: So, yeah. More than one link, matches something in the pool, but not actually linked to it. Isn't that *awesome*? ;'( I very much want BackupPC_fixLinks to deal with this, and I'm trying to modify it to do that now

Re: [BackupPC-users] jLib bug with md5sum stuff

2010-12-06 Thread Robin Lee Powell
On Mon, Dec 06, 2010 at 12:32:01PM -0800, Robin Lee Powell wrote: substr outside of string at /usr/local/lib64/BackupPC/jLib.pm line 162. Use of uninitialized value in concatenation (.) or string at /usr/local/lib64/BackupPC/jLib.pm line 162. The line(s) in question

Re: [BackupPC-users] Bizarre form of cpool corruption.

2010-12-06 Thread Robin Lee Powell
On Mon, Dec 06, 2010 at 10:46:12PM -0800, Craig Barratt wrote: Robin, ls -li /backups/cpool/1/5/c/15c0e4b08058ef3704b8fc24887e2bcc /backups/pc/foo--tm50-e00145--tm50-s00339---shared/47/f%2f/fshared/ffoo/fpurchase_order_assets/fbatch_7813/f7105620_done.txt *BUT*. Not linked. $ ls

Re: [BackupPC-users] Bizarre form of cpool corruption.

2010-12-06 Thread Robin Lee Powell
On Mon, Dec 06, 2010 at 11:33:15PM -0800, Craig Barratt wrote: Robin, Nope; I'm a big tab completion user, so I would have seen it. ;) $ ls -li /backups/cpool/1/5/c/15c0e4b08058ef3704b8fc24887e2bcc* 255523133 -rw-r- 3 backuppc backuppc 27 Nov 22 19:33

Re: [BackupPC-users] Bizarre form of cpool corruption.

2010-12-07 Thread Robin Lee Powell
On Tue, Dec 07, 2010 at 01:07:50PM -0500, Jeffrey J. Kosowsky wrote: 2. More importantly, there is no fast way No, there really isn't. that I know of checking whether a pc entry with more than one link is in the pool. You first need to read in the 1st MB of each file, calculate the partial

[BackupPC-users] Feature? Bug? Something. BackupPC_tarPCCopy and fixing links.

2010-12-07 Thread Robin Lee Powell
So, I've made BackupPC_fixLinks do what I want in terms of fixing the problems that caused it to emit hundreds of: Can't find redbubble--tm50-e00145--tm50-s00339---shared/40/f%2f/fshared/fredbubble/fpurchase_order_assets/fbatch_7749/faddresses_dom estic_20101026T01.csv in pool, will copy file

Re: [BackupPC-users] Bizarre form of cpool corruption.

2010-12-07 Thread Robin Lee Powell
On Tue, Dec 07, 2010 at 01:16:32PM -0500, Jeffrey J. Kosowsky wrote: Robin, can you just clarify the context. Did this apparent pool corruption only occur after running BackupPC_tarPCCopy or did it occur in the course of normal backuppc running. I honestly don't know; I've done so much on

Re: [BackupPC-users] jLib bug with md5sum stuff

2010-12-07 Thread Robin Lee Powell
On Tue, Dec 07, 2010 at 01:34:19PM -0500, Jeffrey J. Kosowsky wrote: OK I was confused for a second there since usually the convention is to use '---' as the original code and '+++' as the new code... Sorry, musta put the diff in the wrong order. Well actually, I had found and fixed that bug

Re: [BackupPC-users] Another jLib/fixLinks issue.

2010-12-07 Thread Robin Lee Powell
On Tue, Dec 07, 2010 at 01:58:28PM -0500, Jeffrey J. Kosowsky wrote: Robin Lee Powell wrote at about 15:40:04 -0800 on Monday, December 6, 2010: This is *fascinating*. From the actually-fixing-stuff part of the run, I get: ERROR: tm50-s00292__nfs/68/f%2f/fshared/fthepoint

Re: [BackupPC-users] Don't use rsync -H!! (was Re: copying the pool to a new filesystem?!)

2010-12-07 Thread Robin Lee Powell
On Tue, Dec 07, 2010 at 02:18:51PM -0500, Dan Pritts wrote: umount /var/lib/backuppc dd if=/dev/onedisk of=/dev/someotherdisk bs=1M Only works if you have identical disks, which is hard when you've got a few TiB on a SAN. In practice, rsync -H is the reasonable way to do what you're after,

[BackupPC-users] It's not rsync -H as such, it's BackupPC data.

2010-12-07 Thread Robin Lee Powell
On Tue, Dec 07, 2010 at 11:44:38AM -0800, Robin Lee Powell wrote: On Tue, Dec 07, 2010 at 02:18:51PM -0500, Dan Pritts wrote: In practice, rsync -H is the reasonable way to do what you're after, EXCEPT that there are just too many hard links on a backuppc data store for this to work

Re: [BackupPC-users] Don't use rsync -H!! (was Re: copying the pool to a new filesystem?!)

2010-12-07 Thread Robin Lee Powell
On Tue, Dec 07, 2010 at 06:19:45PM -0600, Les Mikesell wrote: On 12/7/10 5:19 PM, Tyler J. Wagner wrote: On Tue, 2010-12-07 at 14:18 -0500, Dan Pritts wrote: Not exactly an answer to your question, but i would do this: umount /var/lib/backuppc dd if=/dev/onedisk of=/dev/someotherdisk

Re: [BackupPC-users] Don't use rsync -H!! (was Re: copying the pool to a new filesystem?!)

2010-12-07 Thread Robin Lee Powell
On Wed, Dec 08, 2010 at 12:06:40AM -0500, Dan Pritts wrote: If I had that big a pool, I think I'd not be using a single backuppc instance, but that's just me. I'm guessing it works well for you, I'm glad it does. We aren't; we have 6. The *smallest* is about 4 TiB if disk, most of it in use.

Re: [BackupPC-users] Status and stop backup from command line

2010-12-08 Thread Robin Lee Powell
On Wed, Dec 08, 2010 at 05:14:51PM +, Keith Edmunds wrote: I need to be able to stop backups running during the working day. I'm aware of BlackoutPeriods and, mostly, that manages to achieve what I need. However, there are times when, for whatever reason, backups overrun. What I want to do

Re: [BackupPC-users] Filesystem corruption: consistency of the pool

2010-12-09 Thread Robin Lee Powell
On Thu, Dec 09, 2010 at 09:53:25AM +0100, martin f krafft wrote: also sprach Jeffrey J. Kosowsky backu...@kosowsky.org [2010.11.17.0059 +0100]: I wrote two programs that might be helpful here: 1. BackupPC_digestVerify.pl If you use rsync with checksum caching then this program checks

Re: [BackupPC-users] Filesystem corruption: consistency of the pool

2010-12-09 Thread Robin Lee Powell
On Thu, Dec 09, 2010 at 03:03:37PM -0500, Jeffrey J. Kosowsky wrote: Robin Lee Powell wrote at about 11:20:26 -0800 on Thursday, December 9, 2010: On Thu, Dec 09, 2010 at 09:53:25AM +0100, martin f krafft wrote: also sprach Jeffrey J. Kosowsky backu...@kosowsky.org [2010.11.17.0059 +0100

Re: [BackupPC-users] Filesystem corruption: consistency of the pool

2010-12-09 Thread Robin Lee Powell
On Thu, Dec 09, 2010 at 03:15:41PM -0500, Jeffrey J. Kosowsky wrote: Robin Lee Powell wrote at about 12:06:24 -0800 on Thursday, December 9, 2010: On Thu, Dec 09, 2010 at 03:03:37PM -0500, Jeffrey J. Kosowsky wrote: Robin Lee Powell wrote at about 11:20:26 -0800 on Thursday

Re: [BackupPC-users] Bizarre form of cpool corruption.

2010-12-09 Thread Robin Lee Powell
On Thu, Dec 09, 2010 at 12:05:01AM -0500, Jeffrey J. Kosowsky wrote: Anyway here is the diff. I have not had time to check it much beyond verifying that it seems to run -- SO I WOULD TRULY APPRECIATE IT IF YOU CONTINUE TO TEST IT AND GIVE ME FEEDBACK. Also, it would be great if you would let

Re: [BackupPC-users] Another jLib/fixLinks issue.

2010-12-09 Thread Robin Lee Powell
On Tue, Dec 07, 2010 at 02:27:46PM -0500, Jeffrey J. Kosowsky wrote: Robin Lee Powell wrote at about 11:05:46 -0800 on Tuesday, December 7, 2010: On Tue, Dec 07, 2010 at 01:58:28PM -0500, Jeffrey J. Kosowsky wrote: Robin Lee Powell wrote at about 15:40:04 -0800 on Monday, December 6

Re: [BackupPC-users] Another jLib/fixLinks issue.

2010-12-09 Thread Robin Lee Powell
On Thu, Dec 09, 2010 at 12:35:46AM -0500, Jeffrey J. Kosowsky wrote: Jeffrey J. Kosowsky wrote at about 13:58:28 -0500 on Tuesday, December 7, 2010: Robin Lee Powell wrote at about 15:40:04 -0800 on Monday, December 6, 2010: This is *fascinating*. From the actually-fixing

Re: [BackupPC-users] Bizarre form of cpool corruption.

2010-12-09 Thread Robin Lee Powell
On Thu, Dec 09, 2010 at 06:41:22PM -0500, Jeffrey J. Kosowsky wrote: Robin Lee Powell wrote at about 15:24:30 -0800 on Thursday, December 9, 2010: On Thu, Dec 09, 2010 at 12:05:01AM -0500, Jeffrey J. Kosowsky wrote: Anyway here is the diff. I have not had time to check it much beyond

Re: [BackupPC-users] Migrating backup machines

2010-12-15 Thread Robin Lee Powell
On Wed, Dec 15, 2010 at 04:26:07PM +0100, d.davo...@mastertraining.it wrote: OK, I know that it's an old topic :) I checked the mailing list and wiki but still I can't find the right direction. I just need migrate to a new Backupc server. I don't need to move the pool because my data are

Re: [BackupPC-users] Seed Copy

2010-12-20 Thread Robin Lee Powell
On Mon, Dec 20, 2010 at 10:06:53AM -0500, barry wrote: Looking for a good way to do a seed copy import into backuppc. I want to be able to get the bulk of backup data on a usb disk then import it into backuppc from said disk and resume rsyncs without copying all the data in over the wire. Is

[BackupPC-users] Double-checking something: cpool and links.

2011-01-04 Thread Robin Lee Powell
Am I correct in my belief that: find /backups/cpool/ -links 1 -ls should always return nothing, and that I can freely delete anything it *does* find? -Robin -- http://singinst.org/ : Our last, best hope for a fantastic future. Lojban (http://www.lojban.org/): The language in which this

Re: [BackupPC-users] aborted by signal=PIPE

2011-01-10 Thread Robin Lee Powell
On Mon, Jan 10, 2011 at 01:11:50PM +0330, mohammad tayebi wrote: *Hi Backuppc Users* i have problem ? my backupc server has Raid 5 And mounted /var/lib/backuppc ofcource my quesition is : This Log * 2011-01-09 20:00:03 Got fatal error during xfer (aborted by signal=PIPE)* backuppc

Re: [BackupPC-users] aborted by signal=PIPE

2011-01-10 Thread Robin Lee Powell
On Mon, Jan 10, 2011 at 10:43:50AM -0600, Les Mikesell wrote: On 1/10/2011 10:26 AM, Robin Lee Powell wrote: On Mon, Jan 10, 2011 at 01:11:50PM +0330, mohammad tayebi wrote: *Hi Backuppc Users* i have problem ? my backupc server has Raid 5 And mounted /var/lib/backuppc ofcource my

Re: [BackupPC-users] Is there any danger to removing pool entries with nlink=1

2011-01-20 Thread Robin Lee Powell
On Thu, Jan 20, 2011 at 11:34:01AM -0500, Jeffrey J. Kosowsky wrote: If BackupPC is *not* running, is there any danger to removing pool entries with just one hard link? If there are no new/ directories, that should be fine. -Robin -- http://singinst.org/ : Our last, best hope for a

Re: [BackupPC-users] Is there any danger to removing pool entries with nlink=1

2011-01-20 Thread Robin Lee Powell
On Thu, Jan 20, 2011 at 04:50:47PM -0500, Jeffrey J. Kosowsky wrote: Robin Lee Powell wrote at about 09:52:30 -0800 on Thursday, January 20, 2011: On Thu, Jan 20, 2011 at 11:34:01AM -0500, Jeffrey J. Kosowsky wrote: If BackupPC is *not* running, is there any danger to removing pool

[BackupPC-users] Clients in the backuground queue that don't actually need backing up?

2011-02-01 Thread Robin Lee Powell
I have one server where the background queue is *huge* (hundreds of hosts, small segment below with actual host names trimmed, sorry) backupType = auto,reqTime = 1296592200,user = BackupPC,host = nik[snip] backupType = auto,reqTime = 1296592200,user = BackupPC,host = med[snip] backupType =

Re: [BackupPC-users] Why is the backup not being queued?

2011-02-02 Thread Robin Lee Powell
On Wed, Feb 02, 2011 at 04:07:41PM +0100, martin f krafft wrote: Hello, I have a BackupPC configured with FullPeriod at 7.97 and daily incremental backups, no blackout periods, hourly wakeup, and 2 consecutive backups permitted. One of the hosts' last full backup is now 8.2 days old,

Re: [BackupPC-users] Why is the backup not being queued?

2011-02-02 Thread Robin Lee Powell
On Wed, Feb 02, 2011 at 04:54:57PM +0100, martin f krafft wrote: also sprach Robin Lee Powell rlpow...@digitalkingdom.org [2011.02.02.1628 +0100]: What could be the reason for this? Most likely the disk is too full. Nope. The disk is at 67%. And if it were too full, I would have

[BackupPC-users] General questions about the background queue and link commands.

2011-02-02 Thread Robin Lee Powell
Let me ask some more general questions: What does the background queue actually *mean*? What does a green background in the Host Summary mean? All but one host (!), 226 out of 227, is in the background queue, which seems rather excessive since about half my hosts have recent backups. All the

Re: [BackupPC-users] Why is the backup not being queued?

2011-02-02 Thread Robin Lee Powell
On Wed, Feb 02, 2011 at 05:11:50PM +0100, martin f krafft wrote: also sprach Robin Lee Powell rlpow...@digitalkingdom.org [2011.02.02.1704 +0100]: What does the host log say? Nothing, really: 2011-02-02 15:00:00 Next wakeup is 2011-02-02 16:00:00 2011-02-02 16:00:00 Next wakeup

Re: [BackupPC-users] Why is the backup not being queued?

2011-02-02 Thread Robin Lee Powell
On Wed, Feb 02, 2011 at 05:53:33PM +0100, martin f krafft wrote: also sprach Robin Lee Powell rlpow...@digitalkingdom.org [2011.02.02.1720 +0100]: 2011-02-02 15:00:00 Next wakeup is 2011-02-02 16:00:00 2011-02-02 16:00:00 Next wakeup is 2011-02-02 17:00:00 2011-02-02 17:00:00

Re: [BackupPC-users] Why is the backup not being queued?

2011-02-02 Thread Robin Lee Powell
On Wed, Feb 02, 2011 at 06:20:15PM +0100, martin f krafft wrote: also sprach Robin Lee Powell rlpow...@digitalkingdom.org [2011.02.02.1757 +0100]: There's a log for each client, which you can see by accessing the client in the GUI (it'll be the top-most LOG File link) and a log

Re: [BackupPC-users] General questions about the background queue and link commands.

2011-02-03 Thread Robin Lee Powell
On Thu, Feb 03, 2011 at 05:25:48PM +0100, Holger Parplies wrote: Hi, Robin Lee Powell wrote on 2011-02-02 08:12:47 -0800 [[BackupPC-users] General questions about the background queue and link commands.]: Let me ask some more general questions: What does the background queue actually

Re: [BackupPC-users] General questions about the background queue and link commands.

2011-02-03 Thread Robin Lee Powell
On Thu, Feb 03, 2011 at 09:22:00AM -0800, Robin Lee Powell wrote: OK, so the question becomes, how do I monitor for general backup queue problems? I've had situations where something gets stuck, like a nightly job, and the queue gets backed up, and I want to detect that. I guess if I could

[BackupPC-users] Re-request: finish all backups and stop.

2011-02-12 Thread Robin Lee Powell
Would be Soo handy. -Robin -- http://singinst.org/ : Our last, best hope for a fantastic future. Lojban (http://www.lojban.org/): The language in which this parrot is dead is ti poi spitaki cu morsi, but this sentence is false is na nei. My personal page:

Re: [BackupPC-users] Re-request: finish all backups and stop.

2011-02-13 Thread Robin Lee Powell
On Sat, Feb 12, 2011 at 10:29:12PM -0600, Les Mikesell wrote: On 2/12/11 9:46 PM, Robin Lee Powell wrote: Would be Soo handy. But what would it mean? And what if one or more of the targets is unavailable and you never stop the retries and thus never finish? I mean finish all

Re: [BackupPC-users] Distributing user configs from a central host?

2011-02-17 Thread Robin Lee Powell
That was *not* sent to the right mailing list. Sorry. -Robin On Thu, Feb 17, 2011 at 06:34:58AM -0800, Robin Lee Powell wrote: I have a central server, that happens to be the puppetmaster, that has various users on it. I would like to copy out their information (name, uid, password, .bashrc

[BackupPC-users] Distributing user configs from a central host?

2011-02-17 Thread Robin Lee Powell
I have a central server, that happens to be the puppetmaster, that has various users on it. I would like to copy out their information (name, uid, password, .bashrc, etc) to all my other hosts, but I want to let the users change their stuff on that host, so I don't want to just stick it in

[BackupPC-users] Backup retention and deletion.

2011-05-13 Thread Robin Lee Powell
So, I had FullKeepCntMin set really high, and I lowered it, and it's been several days, and yet I still have a ton of old fulls lying around. My questions: 1. When and how do deletions of the excess occur? 2. Why aren't they occuring? (in particular, is it possible that I need to completely

Re: [BackupPC-users] Backup retention and deletion.

2011-05-13 Thread Robin Lee Powell
On Fri, May 13, 2011 at 10:47:27PM +0200, Matthias Meyer wrote: Robin Lee Powell wrote: So, I had FullKeepCntMin set really high, and I lowered it, and it's been several days, and yet I still have a ton of old fulls lying around. My questions: 1. When and how do deletions

Re: [BackupPC-users] Backup retention and deletion.

2011-05-13 Thread Robin Lee Powell
On Sat, May 14, 2011 at 04:09:51AM +0200, Holger Parplies wrote: config.pl commented on 2010-07-31 19:52 [BackupPC 3.2.0]: # Note that $Conf{FullAgeMax} will be increased to $Conf{FullKeepCnt} # times $Conf{FullPeriod} if $Conf{FullKeepCnt} specifies enough # full backups to exceed

[BackupPC-users] The one-at-a-time nightly problem returns (was Re: The one-at-a-time nightly problem, debugged; DEVS PLEASE READ; long.)

2011-08-21 Thread Robin Lee Powell
. This is really bad for my systems, and really frustrating. -_- -Robin On Wed, Nov 24, 2010 at 06:11:20PM -0800, Robin Lee Powell wrote: Figured it out. The problem was that I have BackupPC set to run 8 nightlies at once (which usually takes 12 or more hours), but it was ending up in a state

[BackupPC-users] Solved, again; devs please read (was Re: The one-at-a-time nightly problem returns (was Re: The one-at-a-time nightly problem, debugged; DEVS PLEASE READ; long.))

2011-08-21 Thread Robin Lee Powell
start a backup, even a bgQueue backup, when a link for that host is pending. 2. Don't put a backup for a host on the bgQueue when a backup (or link job) for that host is currently running, even in QueueAllPCs. -Robin On Sun, Aug 21, 2011 at 08:52:38AM -0700, Robin Lee Powell wrote: So, I haven't

[BackupPC-users] Feature request: log both starts.

2011-08-21 Thread Robin Lee Powell
We use dumpPreUserCmd to run database dumps, then backup the resulting files. Some of these dumps take a *really* long time. The annoying part is that the logs don't show the *actual* start time of the backups (that is, the time when the dumpPreUserCmd is started), they only show the start of

Re: [BackupPC-users] Solved, again; devs please read (was Re: The one-at-a-time nightly problem returns (was Re: The one-at-a-time nightly problem, debugged; DEVS PLEASE READ; long.))

2011-08-22 Thread Robin Lee Powell
in QueueAllPCs. -Robin On Sun, Aug 21, 2011 at 08:52:38AM -0700, Robin Lee Powell wrote: So, I haven't figured out why yet, but this keeps happening on my hosts *even though* I'm not running BackupPC_serverMesg BackupPC_nightly run. As far as I know, other than BackupPC_serverMesg server reload

<    1   2