On 2014-09-13 08:52:33 +0300, Ants Aasma wrote: > On Sat, Sep 13, 2014 at 6:59 AM, Arthur Silva <arthur...@gmail.com> wrote: > > That's not entirely true. CRC-32C beats pretty much everything with the same > > length quality-wise and has both hardware implementations and highly > > optimized software versions. > > For better or for worse CRC is biased by detecting all single bit > errors, the detection capability of larger errors is slightly > diminished. The quality of the other algorithms I mentioned is also > very good, while producing uniformly varying output.
There's also much more literature about the various CRCs in comparison to some of these hash allgorithms. Pretty much everything tests how well they're suited for hashtables, but that's not really what we need (although it might not hurt *at all* to have something faster there...). I do think we need to think about the types of errors we really have to detect. It's not all that clear that either the typical guarantees/tests for CRCs nor for checksums (smhasher, whatever) are very representative... > CRC has exactly > one hardware implementation in general purpose CPU's and Intel has a > patent on the techniques they used to implement it. The fact that AMD > hasn't yet implemented this instruction shows that this patent is > non-trivial to work around. I think AMD has implemeded SSE4.2 with bulldozer. It's still only recent x86 though. So I think there's good reasons for moving away from it. How one could get patents on exposing hardware CRC implementations - hard to find a computing device without one - as a instruction is beyond me... I think it's pretty clear by now that we should move to lz4 for a couple things - which bundles xxhash with it. So that has one argument for it. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers