I re-ran the benchmark that's had me most worried against the committed code and things look good so far. I've been keeping quiet because my tests recently have all agreed with what Ants already described. This is more a confirmation summary than new data.

The problem case has been Jeff's test 2 "worst-case overhead for calculating checksum while reading data" from the OS cache. I wrapped that into a test harness and gave results similar to Jeff's at http://www.postgresql.org/message-id/5133d732.4090...@2ndquadrant.com
based on the originally proposed Fletcher-16 checksum.

I made some system improvements since then such that the absolute runtime improved for most of the tests I'm running. But the percentage changes didn't seem off enough to bother re-running the Fletcher tests again. Details are in attached spreadsheet, to summarize:

-The original Fletcher-16 code slowed this test case down 24 to 32%, depending on whether you look at the average of 3 runs or the median.

-The initial checksum commit with the truncated WAL CRC was almost an order of magnitude worse: 146% to 224% slowdown. The test case that took ~830ms was taking as much as 2652ms with that method. I'm still not sure why the first run of this test is always so much faster than the second and third. But since it happens so often I think it's fair to consider that worst case really important.

-Committed FNV-1a implementation is now slightly better than Fletcher-16 speed wise: 19 to 27% slowdown.

-Slicing by 8 CRC I didn't test because once I'd fully come around to agree with Ants's position it didn't seem likely to be useful. I don't want to lose track of that idea though, it might be the right path for a future implementation with 32 bit checksums.

Since the >=25% slowdown on this test with Fletcher-16 turned into more like a 2% drop on more mixed workloads, I'd expect we're back to where that's again the case with the new FNV-1a. I plan to step back to looking at more of those cases, but it will take a few days at least to start sorting that out.

Greg Smith   2ndQuadrant US    g...@2ndquadrant.com   Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com

Attachment: ChecksumMethods.xls
Description: MS-Excel spreadsheet

Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:

Reply via email to