Re: [BackupPC-users] BackupPC_verifyPool mismatchs found - what to do?
backu...@kosowsky.org wrote: > Matthias Meyer wrote at about 08:41:34 +0100 on Monday, December 3, 2012: > > Wow - thank you for details :) > > > > Most of the files found are realy top level attrib files. > > But BackupPC_verifyPool.pl some files found like > > [1459587] fXLC_LOCALE ( 979) != > > [c5e2162bfb3286475d4b71503593ffcd > > [1459783] attrib (73) != > > [42d8ce042b950daa935fe4e0440d2020 > > too. > > > > Anyone have an Idea whats happen here? > > Should I simply rename those files as "suggested"? > As mentioned in Holger's and my replies, I would do the following: > > 1. Check to see you have the latest version of BackupPC and check the >changelogs to verify that the error has been corrected. I'm running BackupPC V3.1 "Unfortunately" I've running a patched version ;) to make yearly backups and do not remove partial backups until a successfull backup is finished. So, maybee, it is my fault but I didn't believe that ;) > > 2. Look up the thread I referenced so you can understand what is going >on with the top level attribs. I've read it. Thanks :) I've a lot of PCs with more than one share. Blessedly - it isn't a real problem :) > > 3. Give some more information about the files that are not top-level >attribs so that we can try to understand what is going on. What is >the content? what are their real names? (search the pc-tree) What >types of files? etc. etc. Look for patterns. I make a regulary backup of my backuppc data with rsync to another device. The analyzing performs on this backup-device. 3010068 files in 4096 directories checked, 3538 had wrong digests, of these 2 zero-length. Allways most of the 3538 files I'm found in /var/lib/backuppc/pc and they can be simply renamed. But some files seems to have special problems. e.g. of files with wrong MD5-names as well as my analyzis (so far): 1) I can't find the inode in /var/lib/backuppc/pc. I would believe BackupPC_nightly will remove this file 144312220 -rw-r- 2 backuppc backuppc 3976 20. Dez 2009 /var/lib/backuppc/cpool/0/0/7/00754a6772b82e796d3cc12ac84661~0 2) Hardlink-refCount seems to be wrong. How to correct it? e2fsck doesn't do this job. 157158688 -rw-r- 3 backuppc backuppc 346 20. Dez 2009 /var/lib/backuppc/cpool/0/0/3/0035b696f4c9aa129dd310fbac63db~0 157158688: (3) 4kB 20.Dez.2009 /var/lib/backuppc/pc/vdr/245/f%2f/fusr/fshare/fdoc/fdefoma/fcopyright 3) the name in cpool isn't a MD5 a) In which case this can be happen? If BackupPC_link has been interrupted? b) realy a Hardlink-refCount of 10003? The backuppc status page say: "Pool hashing gives 3474 repeated files with longest chain 31," 152963304 -rw-r- 10003 backuppc backuppc 86 13. Dez 2011 /var/lib/backuppc/cpool/0/1/4/fcommon 152963304: (10003) 4kB 13.Dez.2011 /var/lib/backuppc/pc/vdr/645/froot/fusr/fshare/fdoc/fkde/fHTML/fda/fkleopatra/fcommon 152963304: (10003) 4kB 13.Dez.2011 /var/lib/backuppc/pc/vdr/645/froot/fusr/fshare/fdoc/fkde/fHTML/fda/fkwatchgnupg/fcommon 152963304: (10003) 4kB 13.Dez.2011 /var/lib/backuppc/pc/vdr/645/froot/fusr/fshare/fdoc/fkde/fHTML/fda/fknode/fcommon 152963304: (10003) 4kB 13.Dez.2011 /var/lib/backuppc/pc/vdr/645/froot/fusr/fshare/fdoc/fkde/fHTML/fda/fkjots/fcommon 152963304: (10003) 4kB 13.Dez.2011 /var/lib/backuppc/pc/vdr/645/froot/fusr/fshare/fdoc/fkde/fHTML/fda/fkorganizer/fcommon 4) another example of wrong Hardlink-refCount together with wrong MD5 filename 62523004 -rw-r- 9 backuppc backuppc 12662 14. Dez 2009 /var/lib/backuppc/cpool/0/2/2/fshadow.mo 62523004: (9) 16kB 14.Dez.2009 /var/lib/backuppc/pc/vdr/406/f%2f/fusr/fshare/flocale/fid/fLC_MESSAGES/fshadow.mo 62523004: (9) 16kB 14.Dez.2009 /var/lib/backuppc/pc/vdr/415/f%2f/fusr/fshare/flocale/fid/fLC_MESSAGES/fshadow.mo 62523004: (9) 16kB 14.Dez.2009 /var/lib/backuppc/pc/vdr/245/f%2f/fusr/fshare/flocale/fid/fLC_MESSAGES/fshadow.mo 62523004: (9) 16kB 14.Dez.2009 /var/lib/backuppc/pc/vdr/381/f%2f/fusr/fshare/flocale/fid/fLC_MESSAGES/fshadow.mo 62523004: (9) 16kB 14.Dez.2009 /var/lib/backuppc/pc/vdr/443/f%2f/fusr/fshare/flocale/fid/fLC_MESSAGES/fshadow.mo 62523004: (9) 16kB 14.Dez.2009 /var/lib/backuppc/pc/vdr/440/f%2f/fusr/fshare/flocale/fid/fLC_MESSAGES/fshadow.mo 62523004: (9) 16kB 14.Dez.2009 /var/lib/backuppc/pc/vhost/427/f%2f/fusr/fshare/flocale/fid/fLC_MESSAGES/fshadow.mo > > 4. As Holger mentioned, there is no harm not renaming other than >wasted pool space (which is typically quite small for attrib files) >and the inelegance of having wrong md5sum file names. Otherwise, >you can use my routine to fix them automatically or you can write >your own or do it manually. I will use your script. But first I'ld like to understand what's happend :) Also interesting. All files from 2012, found by Holgers script, are top level attrib files. The strange
Re: [BackupPC-users] BackupPC_verifyPool mismatchs found - what to do?
Matthias Meyer wrote at about 08:41:34 +0100 on Monday, December 3, 2012: > Wow - thank you for details :) > > Most of the files found are realy top level attrib files. > But BackupPC_verifyPool.pl some files found like > [1459587] fXLC_LOCALE ( 979) != > c5e2162bfb3286475d4b71503593ffcd > [1459783] attrib (73) != > 42d8ce042b950daa935fe4e0440d2020 > too. > > Anyone have an Idea whats happen here? > Should I simply rename those files as "suggested"? As mentioned in Holger's and my replies, I would do the following: 1. Check to see you have the latest version of BackupPC and check the changelogs to verify that the error has been corrected. 2. Look up the thread I referenced so you can understand what is going on with the top level attribs. 3. Give some more information about the files that are not top-level attribs so that we can try to understand what is going on. What is the content? what are their real names? (search the pc-tree) What types of files? etc. etc. Look for patterns. 4. As Holger mentioned, there is no harm not renaming other than wasted pool space (which is typically quite small for attrib files) and the inelegance of having wrong md5sum file names. Otherwise, you can use my routine to fix them automatically or you can write your own or do it manually. -- Keep yourself connected to Go Parallel: BUILD Helping you discover the best ways to construct your parallel projects. http://goparallel.sourceforge.net ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] BackupPC_verifyPool mismatchs found - what to do?
Wow - thank you for details :) Most of the files found are realy top level attrib files. But BackupPC_verifyPool.pl some files found like [1459587] fXLC_LOCALE ( 979) != c5e2162bfb3286475d4b71503593ffcd [1459783] attrib (73) != 42d8ce042b950daa935fe4e0440d2020 too. Anyone have an Idea whats happen here? Should I simply rename those files as "suggested"? br Matthias -- Don't Panic -- Keep yourself connected to Go Parallel: BUILD Helping you discover the best ways to construct your parallel projects. http://goparallel.sourceforge.net ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] BackupPC_verifyPool mismatchs found - what to do?
Holger Parplies wrote at about 17:59:43 +0100 on Sunday, December 2, 2012:
> Hi,
>
> Matthias Meyer wrote on 2012-12-02 01:57:07 +0100 [[BackupPC-users]
> BackupPC_verifyPool mismatchs found - what to do?]:
> > I've tried the BackupPC_verifyPool.pl from Holger Parplies.
> > Unfortunately some MD5 errors found and print out. e.g.:
> > [28878] 083212b41c2482783128c9212f1f8a26 (78) !=
> > 70f6bdb839eed7efdfe8f8b01f4dcbc7
> >
> > Did I have a problem?
>
> if there are no other indications, not necessarily. While this is not
> *supposed* to happen, there used to be a bug in BackupPC that caused
> top-level
> attrib files for backups with more than one share to be linked into the pool
> with an incorrect pool file name. The only adverse effect was that the (small
> amount of) space for the attrib file was not shared between backups. I'm not
> sure when this bug was fixed. Perhaps Jeffrey can provide you with more
> detail
> - he's the one that debugged the problem. The indicated file size (78 bytes)
> seems plausible for a top-level attrib file, so this may well be what you are
> seeing.
Here is a link to the thread explaining the original problem and
outlining several possible solutions.
http://adsm.org/lists/html/BackupPC-users/2009-12/msg00259.html
I think that Craig ultimately corrected it in the current release but
I am not sure.
Note that I have also diagnosed problems when switching between x86
and ARM architectures due to bugs in the checkusmming algorithms
though I would guess this is unlikely to be your issue. In addition to
being an unlikely use case, it also happens to make all the file
checksums wrong.
>
> > What should/can I do?
>
> Investigate whether this is the case. If not, look at the files in question.
> See below.
>
> > How to find out which file it is?
>
> The above file is cpool/0/8/3/083212b41c2482783128c9212f1f8a26 (or pool/...
> if
> you used the -u switch). Which file(s) in the pc/ tree this links to is more
> difficult to determine. First of all, for the problem I mentioned above, my
> understanding is that the link count of the pool file should be 2 (one pool
> link, one attrib file link; after that the pool file will never be re-used,
> because it will never match, as it has a name not matching its contents). If
> the link count is *not* 2, this seems to be an indication that the contents
> changed on disk when they in no case should have, which is Not Good(tm).
>
> >From the pool file ('ls -i cpool/0/8/3/083212b41c2482783128c9212f1f8a26')
> >you
> can determine the inode and search for that in the pc/ tree. Assuming this is
> a top-level attrib file, you can speed things up greatly by not traversing
> the
> backups. Depending on the number of hosts and backups, you might get away
> with
> something as simple (though not necessarily as fast as you might expect) as
> 'ls -i1 pc/*/*/attrib | sort > /tmp/top-level-attrib-inums'. Search the
> output
> file for the inode numbers you determined.
>
> If it's not top-level attrib files, you're looking at something like
> 'find pc/ -inum ... -ls', which will take *long*. You'll probably want to at
> least look for all inodes in one traversal, which is probably easier to code
> in Perl than type into one find invocation ;-).
>
> In any case, you can look at the contents using just the pool file name and
> BackupPC_zcat. That might give you enough information to be able to locate
> the
> file.
> For attrib files, BackupPC_zcat produces output that is not very human
> readable, though it *does* contain the file names (meaning share names, in
> the
> case of a top-level attrib file), so it might be good enough. I'm not sure
> whether BackupPC_attribPrint will work with the pool file name, but you could
> try that as well.
>
> Hope that helps.
>
> Regards,
> Holger
Thanks Holger for giving the above pointers.
While I am not familiar with Holger's routine, I have written a
similar routine that verifies and also *fixes* broken md5sum file
names. It does so by re-naming +/- moving the pool file as required.
Note that the program is more complicated than one might expect since
it needs to deal with "chains" of pool files with the same partial
md5sum but different suffixes, including knowing where/when to add into
the chain, including linking rather than copying if the target already
exists in the chain or alternatively adding a new suffix to the chain if the
content is distinct or
if MAXLINKS exceeded. Also, if the moved file is part of a chain, then
the original chain needs to be renumbered to fill
Re: [BackupPC-users] BackupPC_verifyPool mismatchs found - what to do?
Hi,
Matthias Meyer wrote on 2012-12-02 01:57:07 +0100 [[BackupPC-users]
BackupPC_verifyPool mismatchs found - what to do?]:
> I've tried the BackupPC_verifyPool.pl from Holger Parplies.
> Unfortunately some MD5 errors found and print out. e.g.:
> [28878] 083212b41c2482783128c9212f1f8a26 (78) !=
> 70f6bdb839eed7efdfe8f8b01f4dcbc7
>
> Did I have a problem?
if there are no other indications, not necessarily. While this is not
*supposed* to happen, there used to be a bug in BackupPC that caused top-level
attrib files for backups with more than one share to be linked into the pool
with an incorrect pool file name. The only adverse effect was that the (small
amount of) space for the attrib file was not shared between backups. I'm not
sure when this bug was fixed. Perhaps Jeffrey can provide you with more detail
- he's the one that debugged the problem. The indicated file size (78 bytes)
seems plausible for a top-level attrib file, so this may well be what you are
seeing.
> What should/can I do?
Investigate whether this is the case. If not, look at the files in question.
See below.
> How to find out which file it is?
The above file is cpool/0/8/3/083212b41c2482783128c9212f1f8a26 (or pool/... if
you used the -u switch). Which file(s) in the pc/ tree this links to is more
difficult to determine. First of all, for the problem I mentioned above, my
understanding is that the link count of the pool file should be 2 (one pool
link, one attrib file link; after that the pool file will never be re-used,
because it will never match, as it has a name not matching its contents). If
the link count is *not* 2, this seems to be an indication that the contents
changed on disk when they in no case should have, which is Not Good(tm).
>From the pool file ('ls -i cpool/0/8/3/083212b41c2482783128c9212f1f8a26') you
can determine the inode and search for that in the pc/ tree. Assuming this is
a top-level attrib file, you can speed things up greatly by not traversing the
backups. Depending on the number of hosts and backups, you might get away with
something as simple (though not necessarily as fast as you might expect) as
'ls -i1 pc/*/*/attrib | sort > /tmp/top-level-attrib-inums'. Search the output
file for the inode numbers you determined.
If it's not top-level attrib files, you're looking at something like
'find pc/ -inum ... -ls', which will take *long*. You'll probably want to at
least look for all inodes in one traversal, which is probably easier to code
in Perl than type into one find invocation ;-).
In any case, you can look at the contents using just the pool file name and
BackupPC_zcat. That might give you enough information to be able to locate the
file.
For attrib files, BackupPC_zcat produces output that is not very human
readable, though it *does* contain the file names (meaning share names, in the
case of a top-level attrib file), so it might be good enough. I'm not sure
whether BackupPC_attribPrint will work with the pool file name, but you could
try that as well.
Hope that helps.
Regards,
Holger
--
Keep yourself connected to Go Parallel:
DESIGN Expert tips on starting your parallel project right.
http://goparallel.sourceforge.net/
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
[BackupPC-users] BackupPC_verifyPool mismatchs found - what to do?
Hi, I've tried the BackupPC_verifyPool.pl from Holger Parplies. Unfortunately some MD5 errors found and print out. e.g.: [28878] 083212b41c2482783128c9212f1f8a26 (78) != 70f6bdb839eed7efdfe8f8b01f4dcbc7 Did I have a problem? What should/can I do? How to find out which file it is? Thanks in advance Matthias -- Don't Panic -- Keep yourself connected to Go Parallel: DESIGN Expert tips on starting your parallel project right. http://goparallel.sourceforge.net/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
