>
> Would appending the file size to the md5 (either literally or
> notionally) further decrease the astronomically small chance of a
> non-purposely constructed collision?
I don't think so. Also, it would also cause the digests to be different
from the full-file digests in rsync 3.X. (In BackupPC 3.x, adding the size
helped because only a subset of the file contents were used for files
larger than 256K. However, I regret adding the file size at the start of
the digest calculation, not the end!)
Also, you note a number of features that should significantly speed up
> backups (including rsync 30, c-code, --checksum flag, etc.). Have you
> done any benchmarking relative to v3.x?
It depends. For slow clients, and initial backups (which involves a lot of
compression cpu time), there isn't much difference. On faster clients I
have seen significant speed ups. Other than initial compression, I suspect
the server load is significantly lower that 3.X, but I haven't made
measurements.
One place where 4.x is slower is that it doesn't implement block checksum
caching. So if you go back to --ignore-times fulls, there is a lot more
work to do on the server compared to 3.X. That case offsets some or all of
the other performance gains. I haven't decided whether it is worth
implementing block checksum caching because --checksum is quite convenient.
Here are two examples using --checksum fulls:
- A fast client (MacbookPro with a flash drive): a full is about 3x
faster, an incremental is about 4x faster. An initial backup with an empty
pool is no faster. An initial backup with a populated pool is maybe 30%
faster (since --checksum allows any pool file to be matched), with a big
saving in network traffic. The reported BackupPC speed is >100MB/sec for
the full (but obviously not much data is being transferred because of
--checksum).
- Backing up part of my BackupPC server to the same server (an old Xeon
with 3ware RAID): a full is about 30% faster, an incremental is about 3x
faster (another backup was running during these tests).
Craig
------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:
Build for Windows Store.
http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
BackupPC-devel mailing list
BackupPC-devel@lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-devel
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
