Hi Craig, Guillermo, all.
A quick update from my side
I have a (complete?) list of (potentially) broken files as reported by
BTRFS read errors.
I then validated the md5sum from the (c)pool with a one-liner:
for i in $(grep /cpool/ ~/broken_files.txt); do
M=$(/usr/share/backuppc/bin/BackupPC_zcat $i | md5sum | cut -d ' ' -f
1); echo -n "$M: "; echo $i | grep --color $M || echo "$f ERR"; done
--> No error, perhaps I am lucky? :-)
This leaves only one file that might be damaged:
backuppc/pc/abc.def.ghi/1989/refCnt/poolCnt.1.16 ?
I am now experimenting with running
/usr/share/backuppc/bin/BackupPC_refCountUpdate -m
(Until now: No error; exit code 0)
Reading the source I have the following questions:
1.
https://github.com/backuppc/backuppc/blob/master/bin/BackupPC_refCountUpdate#L819
does "rand(100) < $Conf{PoolNightlyDigestCheckPercent}"
My understanding would be that I won't get a predictable 1% check on
*each* run. Perhaps on average, with a sufficiently large pool, over a
longer period of time, this might work out, but for a few runs of
BackupPC_refCountUpdate not.
--> Perhaps there would be another implementation that gets a more
predictable result?
(i.e. create an array of all the files to check, sort them, and then
take the first $Conf{PoolNightlyDigestCheckPercent} of entries, but at
least 1?)
2. I also seem to have accumulated checksum errors over the past years.
--> How do I proceed with these files? If they still exist on the
source, I'd like to re-sync them to the backup.
3. For backuppc/pc/abc.def.ghi/1989/refCnt/poolCnt.1.16 , I do not find
a way to re-check only this one, old backup.
--> Would it be a good idea to add an option to add a "-n num" flag to
BackupPC_refCountUpdate to be able to operate on a particular backup?
Thanks for your guidance,
Raoul
On 2020-08-24 20:39, Raoul Bhatia wrote:
Hi Craig and Guillermo,
On 2020-08-23 19:35, Craig Barratt via BackupPC-users wrote:
$Conf{PoolNightlyDigestCheckPercent} is in percent, so you should set
it to 100 to check all the pool file's MD5 digest against their file
names.
As Guillermo mentions, to check the pool MD5 digests, you can set
temporarily set $Conf{PoolNightlyDigestCheckPercent} to 100 and
$Conf{PoolSizeNightlyUpdatePeriod} to 1.
When reading the documentation, I also came across these options.
However, I didn't dare to run backuppc / BackupPC_nightly, because
from the documentation:
Overnight, when BackupPC_nightly next runs,
all the unused pool files will be deleted and
this will recover the disk space used by the client's backups.
I didn't want to end up with an empty pool...
If you stop BackupPC, to check all the pool digests, run:
BackupPC_refCountUpdate -m
If you want to also regenerate all the host reference counts (which
will take a long time), you could run:
BackupPC_refCountUpdate -m -F
Meanwhile, with the kind help of the btrfs community, I figured out a
way to get the damaged files. This process is not finished, yet,
however, I have a first list:
/mnt/backuppc/pc/abc.def.ghi/1989/refCnt/poolCnt.1.16
/mnt/backuppc/cpool/5c/c8/5cc9373a32e06baaa308a7b341db5ac9
/mnt/backuppc/cpool/b2/62/b3629c46481cb038682aea248c45b89f
/mnt/backuppc/cpool/20/c6/21c61013d40e644af734e28459df0a1a
/mnt/backuppc/cpool/8a/f8/8af935bc53f7199ed75a5695bfd57f26
FYI: The most recent backup from host abc.def.ghi is 2291, so 1989 is
quite far in the past.
How should I proceed when I have a list of broken files?
Move them out from cpool and hope they will be re-synced by the next
(full?) backup?
Thanks,
Raoul
Craig
On Sun, Aug 23, 2020 at 6:45 AM Guillermo Rozas <guille2...@gmail.com>
wrote:
Hi Raoul,
are you using BackupPC v4? If yes, you can use a modification of the
script I posted here:
https://sourceforge.net/p/backuppc/mailman/message/37032497/
In the latest version (4.4.0) you also have the config option
$Conf{PoolNightlyDigestCheckPercent}, which checks the md5 digest of
this fraction of the pool files each night. You can probably set it to
1 and wait a night for it to run.
Regards,
Guillermo
On Sun, Aug 23, 2020 at 5:38 AM Raoul Bhatia <ra...@bhatia.at> wrote:
Hi,
related to my previous email, it seems that the cause of my issues was
a
file system corruption after a "power cut".
I managed to recover (most of?) the data and would now like to do a
thorough check of the data.
Is there any way to "fully verify" the integrity of my backuppc
installation, ideally in a nondestructive way ;-)
Thanks,
Raoul
PS. My backuppc process is stopped.
--
DI (FH) Raoul Bhatia MSc
E-Mail. ra...@bhatia.at
Tel. +43 699 10132530
_______________________________________________
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/