Hello John,
Some of the algorithm are not fully implemented, they are more like place
holders.
In our tests, the best one was XXHASH64, not sure that XXH3 was fully done
as it was less good as XXHASH64.
Maybe the Accurate fileset option is not ready for the check with XXHASH64
or XXH3, it should be available in the next major version.
Do not forget that using the option at this place will read all
files of your system at each incremental backup, modified files will be
read twice.
Best Regards,
Eric
On 4/18/25 22:48, John Lockard wrote:
From various places online I see that these are the options and signature
comparison values for the various hash methods.
1 : SHA1
2 : SHA256
3 : SHA512
5 : MD5
6 : XXHASH64
7 : XXH3_64
8 : XXH3_128
But, then diving into the code (15.0.2) I see that there are issues with
SHA512 and that's disabled in the code, in some places and not in others.
I also see that for XXH3_64 the code says it was never implemented, and
again disabled in some sections of the code, but not others.
So, I'd like to use a Signature of: XXH3_128, that seems to pass the muster
for the "Signature =" line in the Fileset. However in the "Accurate ="
line, the "8" is being rejected, as in "Accurate = pino8" and "Verify =
pins8".
Am I reading the code wrong and "8" is not the value associated with
XXH3_128?
Also, of note, that none of this is updated in the documentation. There
are mentions in the ChangeLog about various things being added, but in the
appropriate section (FileSet Resource) they aren't there, or are completely
wrong.
- XXHASH is mentioned as a Signature option in the docs, yet that
specific string can't be found as an option in the source.
- There is no mention in the docs of XXH3_64 or XXH3_128. In the code
both are acceptable for the Signature, yet they fail validation.
Best I can see is that the options are not included in scan_include_options
within src/dird/inc_conf.c. At line 281:
/* Check if the options are correct */
switch(keyword) {
case INC_KW_VERIFY:
fs_options = "ipnugsamcd5123:V"; /* From dird/verify.c */
break;
case INC_KW_BASEJOB:
case INC_KW_ACCURATE:
fs_options = "oipnugsamMcdA5123:JC"; /* From filed/accurate.c
accurate_check_file() */
break;
default:
break;
I suspect the first fs_options should be:
fs_options = "ipnugsamcd5123678:V"; /* From dird/verify.c */
and, the second fs_options should be:
fs_options = "oipnugsamMcdA5123678:JC"; /* From filed/accurate.c
accurate_check_file() */
Though, in src/filed/accurate.c in the accurate_check_file function, at
line no 640, in this chunk:
/* Go through opts once again, this time check only for checksum-related
opts */
for (char *p=opts; *p; p++) {
switch (*p) {
case '5': /* compare MD5 */
case '1': /* compare SHA1 */
case '2': /* compare SHA256 */
case '3': /* compare SHA512 */
case '6': /* compare XXHASH64 */
case '7': /* compare XXH3_64 */
case '8': /* compare XXH3_128 */
if (ff_pkt->type != FT_LNKSAVED &&
(S_ISREG(ff_pkt->statp.st_mode) &&
ff_pkt->flags & (FO_MD5|FO_SHA1|FO_SHA256|FO_SHA512))) {
checksum = true;
}
break;
case 'o':
only_changed = true;
break;
default:
break;
}
I'm thinking this line:
ff_pkt->flags & (FO_MD5|FO_SHA1|FO_SHA256|FO_SHA512))) {
Is probably supposed to read:
ff_pkt->flags &
(FO_MD5|FO_SHA1|FO_SHA256|FO_SHA512|O_XXHASH64|FO_XXH3_64|FO_XXH3_128)))
{
I'm pondering making these changes and seeing what happens.
-John
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
