apurtell edited a comment on pull request #3244: URL: https://github.com/apache/hbase/pull/3244#issuecomment-844561641
I am redoing microbenchmarks with the latest patch and will update here soon. Improvements have unlocked IO performance improvement from the compression. |Algorithm|Avg WAL append in ms - Before|Avg WAL append in ms - Now| |--|--|--| |None|0.290 (stdev 0.328)|unchanged| |Compression, but no value compression|0.372 (stddev 0.336)|unchanged| |Compression, value compression=SNAPPY|2.971 (stddev 2.029)|**0.0272 (stddev 0.204)**| Before, SNAPPY+copy had a 10x loss, now SNAPPY+no-copy has a 10x gain. I double checked these findings and it seems correct. I will re-run the benchmark when measuring for the other codec types for comparison. Microbenchmarks are collected with [this change](https://gist.github.com/apurtell/596310d08b5ad75cd9677466d36360e4). Statistics are collected over the lifetime of the regionserver and are dumped at end of test at shutdown. Statistics are updated under synchronization but this is done in a way that excludes that overhead from measurement. The normal patch does not contain either the instrumentation or the synchronization point. Nanoseconds are converted to milliseconds for the table. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
