On Tue, Nov 25, 2014 at 2:30 AM, Liu Bo <bo.li....@oracle.com> wrote: > On Mon, Nov 24, 2014 at 11:34:46AM -0800, John Williams wrote: >> For example, Spooky V2 hash is 128 bits and is very fast. It is >> noncryptographic, but it is more than adequate for data checksums. >> >> http://burtleburtle.net/bob/hash/spooky.html >> >> SnapRAID uses this hash, and it runs at about 15 GB/sec on my machine >> (Xeon E3-1270 V2 @ 3.50Ghz) > > Thanks for the suggestion, I'll take a look. > > Btw, it's not in kernel yet, is it?
No, as far as I know, it is not in the kernel. By the way, as for the suggestion of blake2 hash, note that it is much slower than Spooky V2 hash. That is to be expected, since blake2 is a cryptographic hash (even if it is one that is fast relative to other cryptographic hashes) and as a class, cryptographic hashes tend to be an order of magnitude slower than the fastest noncryptographic hashes. The hashes that I would recommend for use with btrfs checksums are: 1) SpookyHash V2 : for 128 bit hashes on 64-bit systems http://burtleburtle.net/bob/hash/spooky.html 2) CityHash : for 256-bit hashes on all systems https://code.google.com/p/cityhash/ 3) Murmur3 :for 128-bit hashes on 32-bit systems (since Spooky and City are not the fastest on most 32-bit systems) https://code.google.com/p/smhasher/wiki/MurmurHash3 All of those are noncryptographic, but they all have good properties that should make them more than adequate for data checksums and dedup usage. For more information, here are some comparisons of fast hash functions (note that these comparisons were written 2 to 3 years ago): http://blog.reverberate.org/2012/01/state-of-hash-functions-2012.html http://research.neustar.biz/tag/spooky-hash/ -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
