On 2014-10-08 22:13:46 +0300, Heikki Linnakangas wrote: > Hmm. So the simple, non-table driven, calculation gives the same result as > using the lookup table using the reflected lookup code. That's expected; the > lookup method is supposed to be the same, just faster. However, using the > "normal" lookup code, but with a "reflected" lookup table, produces the same > result as Postgres' algorithm. Indeed, that's what we do in PostgreSQL. But > AFAICS, that's an incorrect combination. You're supposed to the > non-reflected lookup table with the non-reflected lookup code; you can't mix > and match. > > As far as I can tell, PostgreSQL's so-called CRC algorithm doesn't > correspond to any bit-by-bit CRC variant and polynomial. My math skills are > not strong enough to determine what the consequences of that are. It might > still be a decent checksum. Or not. I couldn't tell if the good error > detection properties of the normal CRC-32 polynomial apply to our algorithm > or not.
Additional interesting datapoints are that hstore and ltree contain the same tables - but properly use the reflected computation. > Thoughts? It clearly seems like a bad idea to continue with this - I don't think anybody here knows which guarantees this gives us. The question is how can we move away from this. There's unfortunately two places that embed PGC32 that are likely to prove problematic when fixing the algorithm: pg_trgm and tsgist both seem to include crc's in their logic in a persistent way. I think we should provide INIT/COMP/FIN_PG32 using the current algorithm for these. If we're switching to a saner computation, we should imo also switch to a better polynom - CRC-32C has better error detection capabilities than CRC32 and is available in hardware. As we're paying the price pf breaking compat anyway... Arguably we could also say that given that there's been little evident problems with the borked computation we could also switch to a much faster hash instead of continuing to use crc... Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
