LinShunKang commented on code in PR #12685:
URL: https://github.com/apache/kafka/pull/12685#discussion_r1271319659
##########
clients/src/main/java/org/apache/kafka/common/utils/Utils.java:
##########
@@ -517,6 +519,50 @@ public static int murmur2(final byte[] data) {
return h;
}
+ /**
+ * Generates 32 bit murmur2 hash from ByteBuffer
+ * @param data ByteBuffer to hash
+ * @return 32 bit hash of the given ByteBuffer
+ */
+ @SuppressWarnings("fallthrough")
+ public static int murmur2(ByteBuffer data) {
Review Comment:
@divijvaidya
I optimized `Utils#murmur2(ByteBuffer)` through this
[commit](https://github.com/apache/kafka/pull/12685/commits/3dec26dadf60c43e53ff8b13ef83206aabf82637),
and here are the results:
```
Benchmark (bytes) (direct) (littleEndian)
(seed) Mode Cnt Score Error Units
MurMurHashBenchmark.byteArrayMurmur2 128 false false
42 thrpt 10 23952.036 ± 234.049 ops/ms
MurMurHashBenchmark.byteArrayMurmur2 128 false true
42 thrpt 10 24006.164 ± 153.647 ops/ms
MurMurHashBenchmark.byteArrayMurmur2 128 true false
42 thrpt 10 23997.223 ± 155.545 ops/ms
MurMurHashBenchmark.byteArrayMurmur2 128 true true
42 thrpt 10 24014.765 ± 169.769 ops/ms
MurMurHashBenchmark.byteArrayMurmur2 256 false false
42 thrpt 10 12314.915 ± 71.558 ops/ms
MurMurHashBenchmark.byteArrayMurmur2 256 false true
42 thrpt 10 12312.495 ± 87.004 ops/ms
MurMurHashBenchmark.byteArrayMurmur2 256 true false
42 thrpt 10 12299.405 ± 85.441 ops/ms
MurMurHashBenchmark.byteArrayMurmur2 256 true true
42 thrpt 10 12289.485 ± 107.814 ops/ms
MurMurHashBenchmark.byteBufferMurmur2 128 false false
42 thrpt 10 23436.309 ± 169.624 ops/ms
MurMurHashBenchmark.byteBufferMurmur2 128 false true
42 thrpt 10 23436.878 ± 93.588 ops/ms
MurMurHashBenchmark.byteBufferMurmur2 128 true false
42 thrpt 10 25995.246 ± 582.841 ops/ms
MurMurHashBenchmark.byteBufferMurmur2 128 true true
42 thrpt 10 28573.727 ± 155.530 ops/ms
MurMurHashBenchmark.byteBufferMurmur2 256 false false
42 thrpt 10 12210.036 ± 69.601 ops/ms
MurMurHashBenchmark.byteBufferMurmur2 256 false true
42 thrpt 10 12184.773 ± 98.403 ops/ms
MurMurHashBenchmark.byteBufferMurmur2 256 true false
42 thrpt 10 14500.743 ± 196.826 ops/ms
MurMurHashBenchmark.byteBufferMurmur2 256 true true
42 thrpt 10 15421.734 ± 113.717 ops/ms
```
After the optimization, the throughput of `Utils#murmur2(ByteBuffer)` in
`HeapByteBuffer` **decreased** by only **2%** compared to
`Utils#murmur2(byte[])`, while in `DirectByteBuffer`, the throughput
**increased** by **8%**.
##########
clients/src/main/java/org/apache/kafka/common/utils/Utils.java:
##########
@@ -517,6 +519,50 @@ public static int murmur2(final byte[] data) {
return h;
}
+ /**
+ * Generates 32 bit murmur2 hash from ByteBuffer
+ * @param data ByteBuffer to hash
+ * @return 32 bit hash of the given ByteBuffer
+ */
+ @SuppressWarnings("fallthrough")
+ public static int murmur2(ByteBuffer data) {
Review Comment:
@divijvaidya
I optimized `Utils#murmur2(ByteBuffer)` through this
[commit](https://github.com/apache/kafka/pull/12685/commits/3dec26dadf60c43e53ff8b13ef83206aabf82637),
and here are the results:
```
Benchmark (bytes) (direct) (littleEndian)
(seed) Mode Cnt Score Error Units
MurMurHashBenchmark.byteArrayMurmur2 128 false false
42 thrpt 10 23952.036 ± 234.049 ops/ms
MurMurHashBenchmark.byteArrayMurmur2 128 false true
42 thrpt 10 24006.164 ± 153.647 ops/ms
MurMurHashBenchmark.byteArrayMurmur2 128 true false
42 thrpt 10 23997.223 ± 155.545 ops/ms
MurMurHashBenchmark.byteArrayMurmur2 128 true true
42 thrpt 10 24014.765 ± 169.769 ops/ms
MurMurHashBenchmark.byteArrayMurmur2 256 false false
42 thrpt 10 12314.915 ± 71.558 ops/ms
MurMurHashBenchmark.byteArrayMurmur2 256 false true
42 thrpt 10 12312.495 ± 87.004 ops/ms
MurMurHashBenchmark.byteArrayMurmur2 256 true false
42 thrpt 10 12299.405 ± 85.441 ops/ms
MurMurHashBenchmark.byteArrayMurmur2 256 true true
42 thrpt 10 12289.485 ± 107.814 ops/ms
MurMurHashBenchmark.byteBufferMurmur2 128 false false
42 thrpt 10 23436.309 ± 169.624 ops/ms
MurMurHashBenchmark.byteBufferMurmur2 128 false true
42 thrpt 10 23436.878 ± 93.588 ops/ms
MurMurHashBenchmark.byteBufferMurmur2 128 true false
42 thrpt 10 25995.246 ± 582.841 ops/ms
MurMurHashBenchmark.byteBufferMurmur2 128 true true
42 thrpt 10 28573.727 ± 155.530 ops/ms
MurMurHashBenchmark.byteBufferMurmur2 256 false false
42 thrpt 10 12210.036 ± 69.601 ops/ms
MurMurHashBenchmark.byteBufferMurmur2 256 false true
42 thrpt 10 12184.773 ± 98.403 ops/ms
MurMurHashBenchmark.byteBufferMurmur2 256 true false
42 thrpt 10 14500.743 ± 196.826 ops/ms
MurMurHashBenchmark.byteBufferMurmur2 256 true true
42 thrpt 10 15421.734 ± 113.717 ops/ms
```
After the optimization, the throughput of `Utils#murmur2(ByteBuffer)` in
`HeapByteBuffer` **decreased** by only **2%** compared to
`Utils#murmur2(byte[])`, while in `DirectByteBuffer`, the throughput
**increased** by **8%**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
