hi Jeffrey,
Am 23.03.2011 um 19:35 schrieb Jeffrey J. Kosowsky:
> Fresel Michal - hi competence e.U. wrote at about 18:59:12 +0100 on
> Wednesday, March 23, 2011:
>> hi
>>
>> i was thinking about the current state of integrity-tests
>> ("RsyncCsumCacheVerifyProb")
>>
>> on compressed files this means
>> # decompress
>> # create checksum of decompressed
>> # compare cached checksums
>>
>
> If checksum caching is on then the checksum is stored at the end of
> the compressed file. So the file does *not* need to be decompressed.
> Technically, the checksum is only added to the file the 2nd time the
> file is encountered which I imagine is probably due to the fact that
> the native rsync algorithm only transmits the block and full file
> checksums when the file already exists (otherwise perhaps only the
> full file checksum is transmitted)
i was writing about "RsyncCsumCacheVerifyProb" not the cache
When rsync checksum caching is enabled (by adding the --checksum-seed=32761
option to $Conf{RsyncArgs}), the cached checksums can be occasionally verified
to make sure the file contents matches the cached checksums. This is to avoid
the risk that disk problems might cause the pool file contents to get
corrupted, but the cached checksums would make BackupPC think that the file
still matches the client.
on compressed files this starts with "decompress ...." as described above?
>> what about:
>> on first run
>> # create checksum of compressed file = checksum_compressed_1
>> # decompress
>> # create checksum of decompressed
>> # compare checksums (of cache)
>> # if checksums (of cache) are the same - recreate checksum of compressed
>> (just in case we had some disk issue meanwile) = checksum_compressed_2
>> # compare checksum_compressed_1 to checksum_compressed_2
>> # if the same - save checksum of compressed file for future use
>>
>> the next run we just need to checksum the compressed file as (the internal
>> cached checksums) were already verified above
>>
>> results in faster rechecks afterwards:
>> -) no need to decompress
>> -) checksum-calculations on compressed files is faster (less MB's to
>> calculate)
>
> Seems like a lot of extra work since Craig's clever implementation
> leverages existing rsync checksum calculations and gives the block and
> full file md4 checksums for free.
it's just for speedup of the following
With checksum caching enabled, there is a risk that should a file's contents in
the pool be corrupted due to a disk problem, but the cached checksums are still
correct, the corruption will not be detected by a full backup, since the file
contents are no longer read and compared. To reduce the chance that this
remains undetected, BackupPC can recheck cached checksums for a fraction of the
files. This fraction is set with the $Conf{RsyncCsumCacheVerifyProb} setting.
The default value of 0.01 means that 1% of the time a file's checksums are
read, the checksums are verified. This reduces performance slightly, but, over
time, ensures that files contents are in sync with the cached checksums.
We just need to checksum the compressed file as the blocks and full-file of
uncompressed are already verified
Greetings
Mike------------------------------------------------------------------------------
Enable your software for Intel(R) Active Management Technology to meet the
growing manageability and security demands of your customers. Businesses
are taking advantage of Intel(R) vPro (TM) technology - will your software
be a part of the solution? Download the Intel(R) Manageability Checker
today! http://p.sf.net/sfu/intel-dev2devmar
_______________________________________________
BackupPC-devel mailing list
BackupPC-devel@lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-devel
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
