dlg99 commented on a change in pull request #856: ISSUE #590 (@bug W-4556980@)
Reduce excessive CPU usage on client side: add crc3?
URL: https://github.com/apache/bookkeeper/pull/856#discussion_r157135685
##########
File path: microbenchmarks/results.txt
##########
@@ -0,0 +1,89 @@
+# on macbook pro
+# Run complete. Total time: 00:21:26
+
+Benchmark
(bufferType) (digest) Mode Cnt Score Error Units
+DigestTypeBenchmark.digestManager
ARRAY_BACKED MAC thrpt 5 8.586 ? 0.556 ops/ms
+DigestTypeBenchmark.digestManager:?gc.alloc.rate
ARRAY_BACKED MAC thrpt 5 0.692 ? 0.045 MB/sec
+DigestTypeBenchmark.digestManager:?gc.alloc.rate.norm
ARRAY_BACKED MAC thrpt 5 88.010 ? 0.002 B/op
+DigestTypeBenchmark.digestManager:?gc.churn.G1_Eden_Space
ARRAY_BACKED MAC thrpt 5 1.615 ? 8.515 MB/sec
+DigestTypeBenchmark.digestManager:?gc.churn.G1_Eden_Space.norm
ARRAY_BACKED MAC thrpt 5 203.670 ? 1073.904 B/op
+DigestTypeBenchmark.digestManager:?gc.churn.G1_Survivor_Space
ARRAY_BACKED MAC thrpt 5 0.096 ? 0.826 MB/sec
+DigestTypeBenchmark.digestManager:?gc.churn.G1_Survivor_Space.norm
ARRAY_BACKED MAC thrpt 5 12.176 ? 104.841 B/op
+DigestTypeBenchmark.digestManager:?gc.count
ARRAY_BACKED MAC thrpt 5 2.000 counts
+DigestTypeBenchmark.digestManager:?gc.time
ARRAY_BACKED MAC thrpt 5 16.000 ms
+DigestTypeBenchmark.digestManager:?stack
ARRAY_BACKED MAC thrpt NaN ---
+DigestTypeBenchmark.digestManager
ARRAY_BACKED CRC32 thrpt 5 445.869 ? 8.161 ops/ms
+DigestTypeBenchmark.digestManager:?gc.alloc.rate
ARRAY_BACKED CRC32 thrpt 5 ? 10?? MB/sec
+DigestTypeBenchmark.digestManager:?gc.alloc.rate.norm
ARRAY_BACKED CRC32 thrpt 5 ? 10?? B/op
+DigestTypeBenchmark.digestManager:?gc.churn.G1_Eden_Space
ARRAY_BACKED CRC32 thrpt 5 0.815 ? 7.022 MB/sec
+DigestTypeBenchmark.digestManager:?gc.churn.G1_Eden_Space.norm
ARRAY_BACKED CRC32 thrpt 5 1.997 ? 17.191 B/op
+DigestTypeBenchmark.digestManager:?gc.count
ARRAY_BACKED CRC32 thrpt 5 1.000 counts
+DigestTypeBenchmark.digestManager:?gc.time
ARRAY_BACKED CRC32 thrpt 5 8.000 ms
+DigestTypeBenchmark.digestManager:?stack
ARRAY_BACKED CRC32 thrpt NaN ---
+DigestTypeBenchmark.digestManager
ARRAY_BACKED CRC32_C thrpt 5 832.499 ? 56.604 ops/ms
+DigestTypeBenchmark.digestManager:?gc.alloc.rate
ARRAY_BACKED CRC32_C thrpt 5 ? 10?? MB/sec
+DigestTypeBenchmark.digestManager:?gc.alloc.rate.norm
ARRAY_BACKED CRC32_C thrpt 5 ? 10?? B/op
+DigestTypeBenchmark.digestManager:?gc.churn.G1_Eden_Space
ARRAY_BACKED CRC32_C thrpt 5 0.816 ? 7.023 MB/sec
+DigestTypeBenchmark.digestManager:?gc.churn.G1_Eden_Space.norm
ARRAY_BACKED CRC32_C thrpt 5 1.064 ? 9.158 B/op
+DigestTypeBenchmark.digestManager:?gc.count
ARRAY_BACKED CRC32_C thrpt 5 1.000 counts
+DigestTypeBenchmark.digestManager:?gc.time
ARRAY_BACKED CRC32_C thrpt 5 8.000 ms
+DigestTypeBenchmark.digestManager:?stack
ARRAY_BACKED CRC32_C thrpt NaN ---
+DigestTypeBenchmark.digestManager
NOT_ARRAY_BACKED MAC thrpt 5 8.367 ? 0.045 ops/ms
+DigestTypeBenchmark.digestManager:?gc.alloc.rate
NOT_ARRAY_BACKED MAC thrpt 5 502.288 ? 2.868 MB/sec
+DigestTypeBenchmark.digestManager:?gc.alloc.rate.norm
NOT_ARRAY_BACKED MAC thrpt 5 65593.694 ? 12.966 B/op
+DigestTypeBenchmark.digestManager:?gc.churn.G1_Eden_Space
NOT_ARRAY_BACKED MAC thrpt 5 539.102 ? 0.215 MB/sec
+DigestTypeBenchmark.digestManager:?gc.churn.G1_Eden_Space.norm
NOT_ARRAY_BACKED MAC thrpt 5 70401.331 ? 403.899 B/op
+DigestTypeBenchmark.digestManager:?gc.churn.G1_Old_Gen
NOT_ARRAY_BACKED MAC thrpt 5 0.006 ? 0.015 MB/sec
+DigestTypeBenchmark.digestManager:?gc.churn.G1_Old_Gen.norm
NOT_ARRAY_BACKED MAC thrpt 5 0.784 ? 1.962 B/op
+DigestTypeBenchmark.digestManager:?gc.count
NOT_ARRAY_BACKED MAC thrpt 5 55.000 counts
+DigestTypeBenchmark.digestManager:?gc.time
NOT_ARRAY_BACKED MAC thrpt 5 200.000 ms
+DigestTypeBenchmark.digestManager:?stack
NOT_ARRAY_BACKED MAC thrpt NaN ---
+DigestTypeBenchmark.digestManager
NOT_ARRAY_BACKED CRC32 thrpt 5 119.933 ? 7.129 ops/ms
+DigestTypeBenchmark.digestManager:?gc.alloc.rate
NOT_ARRAY_BACKED CRC32 thrpt 5 7206.443 ? 430.557 MB/sec
+DigestTypeBenchmark.digestManager:?gc.alloc.rate.norm
NOT_ARRAY_BACKED CRC32 thrpt 5 65648.001 ? 0.001 B/op
+DigestTypeBenchmark.digestManager:?gc.churn.G1_Eden_Space
NOT_ARRAY_BACKED CRC32 thrpt 5 7684.923 ? 509.880 MB/sec
+DigestTypeBenchmark.digestManager:?gc.churn.G1_Eden_Space.norm
NOT_ARRAY_BACKED CRC32 thrpt 5 70005.330 ? 532.187 B/op
+DigestTypeBenchmark.digestManager:?gc.churn.G1_Old_Gen
NOT_ARRAY_BACKED CRC32 thrpt 5 0.158 ? 0.078 MB/sec
+DigestTypeBenchmark.digestManager:?gc.churn.G1_Old_Gen.norm
NOT_ARRAY_BACKED CRC32 thrpt 5 1.439 ? 0.769 B/op
+DigestTypeBenchmark.digestManager:?gc.count
NOT_ARRAY_BACKED CRC32 thrpt 5 784.000 counts
+DigestTypeBenchmark.digestManager:?gc.time
NOT_ARRAY_BACKED CRC32 thrpt 5 2949.000 ms
+DigestTypeBenchmark.digestManager:?stack
NOT_ARRAY_BACKED CRC32 thrpt NaN ---
+DigestTypeBenchmark.digestManager
NOT_ARRAY_BACKED CRC32_C thrpt 5 130.444 ? 3.441 ops/ms
+DigestTypeBenchmark.digestManager:?gc.alloc.rate
NOT_ARRAY_BACKED CRC32_C thrpt 5 7824.727 ? 212.028 MB/sec
+DigestTypeBenchmark.digestManager:?gc.alloc.rate.norm
NOT_ARRAY_BACKED CRC32_C thrpt 5 65552.001 ? 0.001 B/op
Review comment:
@merlimat in unit tests I did not see this log:
if (!Sse42Crc32C.isSupported()) {
LOG.error("Sse42Crc32C is not supported, will use less slower
CRC32C implementation.");
}
so I assume that it won't use pure java implementation.
This is because DoubleByteBuf returns false == hasMemoryAddress() and false
== hasArray()
public static int computeChecksum(ByteBuf payload) {
if (payload.hasMemoryAddress() && (CRC32C_HASH instanceof
Sse42Crc32C)) {
return CRC32C_HASH.calculate(payload.memoryAddress() +
payload.readerIndex(), payload.readableBytes());
} else if (payload.hasArray()) {
return CRC32C_HASH.calculate(payload.array(),
payload.arrayOffset() + payload.readerIndex(),
payload.readableBytes());
} else {
return CRC32C_HASH.calculate(payload.nioBuffer());
}
}
ends up in .nioBuffer() which ends up doing the following:
public ByteBuffer nioBuffer(int index, int length) {
>>>>> ByteBuffer dst = ByteBuffer.allocate(length);
ByteBuf b = Unpooled.wrappedBuffer(dst);
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
