Hi Guillermo, I agree. BackupPC's RefCnt/Fsck is a must IMHO.
A main difference between PoolNightlyDigestCheck and btrfs' builtin
checksumming is that the nightly digest checks proactively scan the data and
will tell you about broken files as soon as they break (well, as soon as the
next check is done after the broke). Btrfs will only complain on retrieval;
perhaps soon, perhaps never. So you need a way to make sure that your archived
data is actually read every now and then, and compared to the checksums for
consistency.
For btrfs, this is done by the "scrub" operation. Which is recommended to be
done regularly.
In my experience, it's slightly lower on the CPU load than BackupPC's check,
but the actual load should be on I/O to the disk, not the CPU. Default CRC32 is
somewhat weaker than the MD5s used by BackupPC, but this is a concern for only
pool collisions, not for data rot. Newer btrfs' features include, e.g., xxhash
checksums, which are not only faster but also more collision resistant than MD5.
I personally use btrfs scrub exclusively (which also check's my non-BackupPC
files on that disk) and disabled NightlyDigestCheck. However, there is one
significant drawback that might or might not be an issue for you: btrfs scrub
is all-or-nothing; it always scans the entire partition and, to be frank, the
"idle priority scheduler" does not play nicely on every system. In contrast,
BackupPC's nightly checks are more clever in that you can scan parts of your
pool every night, to distribute the load on your system.
So, IMHO, the best approach is to have a separate partition/device exclusively
for your pool, use NightlyDigestCheck there, and *disable* (or not enable)
regular btrfs scrub. (You can keep checksumming active nevertheless, it's
extremely low overhead.)
A scrub does oh-so-slightly more, as it also checks metadata checksums and can
repair corrupted blocks if good copies are available - but problems will also
be detected by a BackupPC nightly scan, and corrective action can be taken.
For any of those, make sure you receive notifications by mail, monitoring, or
LEDs above your pillow in case there are errors. Spending a lot of time on
checksumming, reading SMART data, health checks etc. isn't going to help unless
you know about the results.
For ZFS, no clue.
Best,
Alex
On 5/4/21 6:38 PM, Guillermo Rozas wrote:
> One ensures against file system bit rot, the other ensures backup file
> consistency.
>
>
> I would say $Conf{PoolNightlyDigestCheckPercent} = 1 is also a check for bit
> rot, as the only thing it does is to read the file, re-calculate the md5
> checksum, and compares it with its name (which is the md5 calculated at the
> time of writing). It actually says it in the help, "This is check if there
> has been any server file system corruption."
>
> What controls the consistency of the backup are
> $Conf{PoolSizeNightlyUpdatePeriod} and specially $Conf{RefCntFsck}.
>
> Regards,
> Guillermo
>
>
> _______________________________________________
> BackupPC-users mailing list
> BackupPC-users@lists.sourceforge.net
> List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki: https://github.com/backuppc/backuppc/wiki
> Project: https://backuppc.github.io/backuppc/
>
smime.p7s
Description: S/MIME Cryptographic Signature
_______________________________________________ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
