Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG

2011-10-07 Thread Arnold Krille
On Friday 07 October 2011 01:41:45 Holger Parplies wrote: Hi, Les Mikesell wrote on 2011-10-06 18:17:06 -0500 [Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG]: On Thu, Oct 6, 2011 at 5:21 PM, Arnold Krille arn...@arnoldarts.de wrote

Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG

2011-10-07 Thread Tim Fletcher
On Thu, 2011-10-06 at 17:54 +0200, Holger Parplies wrote: To be honest, I would *hope* that only you had these issues and everyone else's backups are fine, i.e. that your hardware and not the BackupPC software was the trigger (though it would probably need some sort of software bug to come up

Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG

2011-10-07 Thread Holger Parplies
Hi, Tim Fletcher wrote on 2011-10-07 10:21:29 +0100 [Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG]: [...] Another system is currently up to a/f/d of a full scan and has found the following errors tim@carbon:~$ grep -v ok /tmp/pool-check.txt

Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG

2011-10-07 Thread Holger Parplies
Hi, Jeffrey J. Kosowsky wrote on 2011-10-07 01:08:03 -0400 [Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG]: Holger Parplies wrote at about 05:46:36 +0200 on Friday, October 7, 2011: Jeffrey J. Kosowsky wrote on 2011-10-06 22:54:44 -0400 [Re

Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG

2011-10-07 Thread Tim Fletcher
On Fri, 2011-10-07 at 10:21 +0100, Tim Fletcher wrote: Another system is currently up to a/f/d of a full scan and has found the following errors The final answer off the server with the larger and older install of backuppc is: 2836949 files in 4096 directories checked, 13 had wrong digests,

Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG

2011-10-06 Thread Tim Fletcher
On Wed, 2011-10-05 at 21:35 -0400, Jeffrey J. Kosowsky wrote: Finally, remember it's possible that many people are having this problem but just don't know it, since the only way one would know would be if one actually computed the partial file md5sums of all the pool files and/or restored

Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG

2011-10-06 Thread Timothy J Massey
Tim Fletcher t...@night-shade.org.uk wrote on 10/06/2011 05:17:03 AM: Do you have a script or series of commands to do this check with? I have access to a couple of backuppc installs of various ages and sizes that I can test. Me too, if it can run in a reasonable amount of time. I'd hate

Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG

2011-10-06 Thread Holger Parplies
Hi, Tim Fletcher wrote on 2011-10-06 10:17:03 +0100 [Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG]: On Wed, 2011-10-05 at 21:35 -0400, Jeffrey J. Kosowsky wrote: Finally, remember it's possible that many people are having this problem but just

Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG

2011-10-06 Thread Timothy J Massey
Holger Parplies wb...@parplies.de wrote on 10/06/2011 11:54:05 AM: If you have switched off compression, you'll have to add a '-u' (though I'm not sure this test makes much sense in that case). Well, then, it won't make much sense in *my* case: I missed that this is unique to compressed

Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG

2011-10-06 Thread Les Mikesell
On Thu, Oct 6, 2011 at 11:56 AM, Timothy J Massey tmas...@obscorp.comwrote: Personally, I feel that compression has no place in backups. Back when we were highly limited in capacity by terrible analog devices (i.e. tape!) I used it from necessity. Now, I just throw bigger hard drives at it

Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG

2011-10-06 Thread Timothy J Massey
Les Mikesell lesmikes...@gmail.com wrote on 10/06/2011 01:21:29 PM: On Thu, Oct 6, 2011 at 11:56 AM, Timothy J Massey tmas...@obscorp.com wrote: Personally, I feel that compression has no place in backups. Back when we were highly limited in capacity by terrible analog devices (i.e.

Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG

2011-10-06 Thread Les Mikesell
On Thu, Oct 6, 2011 at 1:04 PM, Timothy J Massey tmas...@obscorp.comwrote: On Thu, Oct 6, 2011 at 11:56 AM, Timothy J Massey tmas...@obscorp.com wrote: Personally, I feel that compression has no place in backups. Back when we were highly limited in capacity by terrible analog devices

Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG

2011-10-06 Thread Arnold Krille
On Thursday 06 October 2011 20:04:57 Timothy J Massey wrote: Les Mikesell lesmikes...@gmail.com wrote on 10/06/2011 01:21:29 PM: On Thu, Oct 6, 2011 at 11:56 AM, Timothy J Massey tmas...@obscorp.com wrote: Personally, I feel that compression has no place in backups. Back when we were

Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG

2011-10-06 Thread Holger Parplies
Hi, Les Mikesell wrote on 2011-10-06 13:42:09 -0500 [Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG]: On Thu, Oct 6, 2011 at 1:04 PM, Timothy J Massey tmas...@obscorp.comwrote: No, the point of backup is to be able to *restore* as much historical

Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG

2011-10-06 Thread Holger Parplies
Hi, Timothy J Massey wrote on 2011-10-06 12:56:42 -0400 [Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed)?cpool files - WEIRD BUG]: Holger Parplies wrote on 10/06/2011 11:54:05 AM: If you have switched off compression, you'll have to add a '-u' (though I'm not sure

Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG

2011-10-06 Thread Les Mikesell
On Thu, Oct 6, 2011 at 5:21 PM, Arnold Krille arn...@arnoldarts.de wrote: No, it makes perfect sense for backuppc where the point is to keep as much history as possible online in a given space. No, the point of backup is to be able to *restore* as much historical data as possible.  Keeping

Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG

2011-10-06 Thread Les Mikesell
On Thu, Oct 6, 2011 at 5:42 PM, Holger Parplies wb...@parplies.de wrote: [...] With compressible data you increase both capacity and reliability by compressing before storage.   There's no magical difference between the reliability of 'cat' vs 'zcat'.  Either one could fail. the problem, I

Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG

2011-10-06 Thread Jeffrey J. Kosowsky
Holger Parplies wrote at about 17:54:05 +0200 on Thursday, October 6, 2011: Hi, Tim Fletcher wrote on 2011-10-06 10:17:03 +0100 [Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG]: On Wed, 2011-10-05 at 21:35 -0400, Jeffrey J. Kosowsky wrote

Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG

2011-10-06 Thread Holger Parplies
Hi, Les Mikesell wrote on 2011-10-06 18:17:06 -0500 [Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG]: On Thu, Oct 6, 2011 at 5:21 PM, Arnold Krille arn...@arnoldarts.de wrote: No, it makes perfect sense for backuppc where the point is to keep

Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG

2011-10-06 Thread John Rouillard
On Thu, Oct 06, 2011 at 05:54:05PM +0200, Holger Parplies wrote: Try something like BackupPC_verifyPool -s -p to scan the whole pool, or BackupPC_verifyPool -s -p -r 0 to test it on the 0/0/0 - 0/0/f pool subdirectories (-r takes a Perl expression evaluating to an array

Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG

2011-10-06 Thread Holger Parplies
Hi, Jeffrey J. Kosowsky wrote on 2011-10-06 19:28:38 -0400 [Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed)?cpool files - WEIRD BUG]: Holger Parplies wrote at about 17:54:05 +0200 on Thursday, October 6, 2011: [...] Actually, what I would propose [...] would be to test

Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG

2011-10-06 Thread Timothy J Massey
Holger Parplies wb...@parplies.de wrote on 10/06/2011 06:58:06 PM: Don't you feel uncomfortable about deduplication, too, then? After all, it introduces a single point of failure for common data. No. Dedupe is merely a side effect of a filesystem. Dedupe errors are no different than any of

Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG

2011-10-06 Thread Les Mikesell
On Thu, Oct 6, 2011 at 8:06 PM, Timothy J Massey tmas...@obscorp.comwrote: Redundancy is a good thing. (While we're on the subject, I've considered Les' argument that compressed files take less space on the disk and are therefore less likely to be corrupted before. It's true, but like

Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG

2011-10-06 Thread Jeffrey J. Kosowsky
Holger Parplies wrote at about 02:45:56 +0200 on Friday, October 7, 2011: Hi, Jeffrey J. Kosowsky wrote on 2011-10-06 19:28:38 -0400 [Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed)?cpool files - WEIRD BUG]: Holger Parplies wrote at about 17:54:05 +0200 on Thursday

Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG

2011-10-06 Thread Holger Parplies
Hi, Jeffrey J. Kosowsky wrote on 2011-10-06 22:09:52 -0400 [Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG]: Holger Parplies wrote at about 02:45:56 +0200 on Friday, October 7, 2011: Jeffrey J. Kosowsky wrote on 2011-10-06 19:28:38 -0400 [Re

Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG

2011-10-06 Thread Jeffrey J. Kosowsky
Jeffrey J. Kosowsky wrote at about 18:58:51 -0400 on Tuesday, October 4, 2011: After the recent thread on bad md5sum file names, I ran a check on all my 1.1 million cpool files to check whether the md5sum file names are correct. I got a total of 71 errors out of 1.1 million files: - 3

Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG

2011-10-06 Thread Holger Parplies
Hi, Jeffrey J. Kosowsky wrote on 2011-10-06 22:54:44 -0400 [Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool?files - WEIRD BUG]: OK... this is a little weird maybe... [...] On all (saved) backups, up to backup 82, the file (and the corresponding cpool file e/f/0

Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG

2011-10-06 Thread Holger Parplies
Hi, I wrote on 2011-10-07 05:46:36 +0200 [Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG]: [...] So far, *none* of my 1.1 million pool files (875000 checked so far) seem to be empty. 1148398 files in 4096 directories checked, 0 zero-length with, 0

Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG

2011-10-06 Thread Jeffrey J. Kosowsky
Holger Parplies wrote at about 05:46:36 +0200 on Friday, October 7, 2011: Hi, Jeffrey J. Kosowsky wrote on 2011-10-06 22:54:44 -0400 [Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool?files - WEIRD BUG]: OK... this is a little weird maybe... [...] On all

Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG

2011-10-05 Thread Holger Parplies
Hi, Jeffrey J. Kosowsky wrote on 2011-10-04 18:58:51 -0400 [[BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG]: After the recent thread on bad md5sum file names, I ran a check on all my 1.1 million cpool files to check whether the md5sum file names

Re: [BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG

2011-10-05 Thread Jeffrey J. Kosowsky
Holger Parplies wrote at about 17:41:48 +0200 on Wednesday, October 5, 2011: Hi, Jeffrey J. Kosowsky wrote on 2011-10-04 18:58:51 -0400 [[BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG]: After the recent thread on bad md5sum file names, I ran

[BackupPC-users] Bad md5sums due to zero size (uncompressed) cpool files - WEIRD BUG

2011-10-04 Thread Jeffrey J. Kosowsky
After the recent thread on bad md5sum file names, I ran a check on all my 1.1 million cpool files to check whether the md5sum file names are correct. I got a total of 71 errors out of 1.1 million files: - 3 had data in it (though each file was only a few hundred bytes long) - 68 of the 71 were