merlimat opened a new pull request, #3309: URL: https://github.com/apache/bookkeeper/pull/3309
### Motivation We are trying to use JNI based implementation of CRC32c and then falling back to a pure Java-based implementation when the shared library is not available. This is typically the case when not running on Linux-x86_64 (eg: macos or linux-arm). When that happens, the CRC performance is much slower. Since Java 9, a new CRC32C class with intrinsic implementation optimized for modern CPUs was added. We should try to use that if we're running on Java >= 9. The JNI version still appears to be slightly faster, so we can keep it the preferred one. ### Microbenchmarks #### Linux ``` Linux - x86_64 ## (Java 9) Benchmark (bufferType) (digest) (entrySize) Mode Cnt Score Error Units DigestTypeBenchmark.digestManager BYTE_BUF_DEFAULT_ALLOC CRC32_C 64 thrpt 5 16316.965 ± 143.502 ops/ms DigestTypeBenchmark.digestManager BYTE_BUF_DEFAULT_ALLOC CRC32_C 1024 thrpt 5 7838.799 ± 134.230 ops/ms DigestTypeBenchmark.digestManager BYTE_BUF_DEFAULT_ALLOC CRC32_C 4086 thrpt 5 4623.477 ± 35.525 ops/ms DigestTypeBenchmark.digestManager BYTE_BUF_DEFAULT_ALLOC CRC32_C 8192 thrpt 5 2033.233 ± 9.059 ops/ms DigestTypeBenchmark.digestManager BYTE_BUF_DEFAULT_ALLOC CRC32_C 16384 thrpt 5 1106.859 ± 11.011 ops/ms DigestTypeBenchmark.digestManager BYTE_BUF_DEFAULT_ALLOC CRC32_C 65536 thrpt 5 277.590 ± 1.651 ops/ms ## (JNI) Benchmark (bufferType) (digest) (entrySize) Mode Cnt Score Error Units DigestTypeBenchmark.digestManager BYTE_BUF_DEFAULT_ALLOC CRC32_C 64 thrpt 2 22237.519 ops/ms DigestTypeBenchmark.digestManager BYTE_BUF_DEFAULT_ALLOC CRC32_C 1024 thrpt 2 12811.964 ops/ms DigestTypeBenchmark.digestManager BYTE_BUF_DEFAULT_ALLOC CRC32_C 4086 thrpt 2 4650.226 ops/ms DigestTypeBenchmark.digestManager BYTE_BUF_DEFAULT_ALLOC CRC32_C 8192 thrpt 2 2726.004 ops/ms DigestTypeBenchmark.digestManager BYTE_BUF_DEFAULT_ALLOC CRC32_C 16384 thrpt 2 1422.693 ops/ms DigestTypeBenchmark.digestManager BYTE_BUF_DEFAULT_ALLOC CRC32_C 65536 thrpt 2 371.433 ops/ms ## (Pure Java) Benchmark (bufferType) (digest) (entrySize) Mode Cnt Score Error Units DigestTypeBenchmark.digestManager BYTE_BUF_DEFAULT_ALLOC CRC32_C 64 thrpt 2 7425.350 ops/ms DigestTypeBenchmark.digestManager BYTE_BUF_DEFAULT_ALLOC CRC32_C 1024 thrpt 2 665.653 ops/ms DigestTypeBenchmark.digestManager BYTE_BUF_DEFAULT_ALLOC CRC32_C 4086 thrpt 2 164.865 ops/ms DigestTypeBenchmark.digestManager BYTE_BUF_DEFAULT_ALLOC CRC32_C 8192 thrpt 2 83.462 ops/ms DigestTypeBenchmark.digestManager BYTE_BUF_DEFAULT_ALLOC CRC32_C 16384 thrpt 2 40.859 ops/ms DigestTypeBenchmark.digestManager BYTE_BUF_DEFAULT_ALLOC CRC32_C 65536 thrpt 2 10.299 ops/ms ``` #### MacOS ``` MacOS - M1 ## (Java 9) Benchmark (bufferType) (digest) (entrySize) Mode Cnt Score Error Units DigestTypeBenchmark.digestManager BYTE_BUF_DEFAULT_ALLOC CRC32_C 64 thrpt 5 66832.647 ± 48152.372 ops/ms DigestTypeBenchmark.digestManager BYTE_BUF_DEFAULT_ALLOC CRC32_C 1024 thrpt 5 15610.884 ± 1760.467 ops/ms DigestTypeBenchmark.digestManager BYTE_BUF_DEFAULT_ALLOC CRC32_C 4086 thrpt 5 3916.049 ± 173.029 ops/ms DigestTypeBenchmark.digestManager BYTE_BUF_DEFAULT_ALLOC CRC32_C 8192 thrpt 5 1992.028 ± 23.020 ops/ms DigestTypeBenchmark.digestManager BYTE_BUF_DEFAULT_ALLOC CRC32_C 16384 thrpt 5 992.235 ± 19.847 ops/ms DigestTypeBenchmark.digestManager BYTE_BUF_DEFAULT_ALLOC CRC32_C 65536 thrpt 5 249.746 ± 1.956 ops/ms ## (Pure Java) Benchmark (bufferType) (digest) (entrySize) Mode Cnt Score Error Units DigestTypeBenchmark.digestManager BYTE_BUF_DEFAULT_ALLOC CRC32_C 64 thrpt 5 8348.394 ± 407.650 ops/ms DigestTypeBenchmark.digestManager BYTE_BUF_DEFAULT_ALLOC CRC32_C 1024 thrpt 5 584.920 ± 13.489 ops/ms DigestTypeBenchmark.digestManager BYTE_BUF_DEFAULT_ALLOC CRC32_C 4086 thrpt 5 147.512 ± 4.767 ops/ms DigestTypeBenchmark.digestManager BYTE_BUF_DEFAULT_ALLOC CRC32_C 8192 thrpt 5 73.776 ± 4.113 ops/ms DigestTypeBenchmark.digestManager BYTE_BUF_DEFAULT_ALLOC CRC32_C 16384 thrpt 5 36.136 ± 1.937 ops/ms DigestTypeBenchmark.digestManager BYTE_BUF_DEFAULT_ALLOC CRC32_C 65536 thrpt 5 9.131 ± 0.157 ops/ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
