[
https://issues.apache.org/jira/browse/CASSANDRA-16360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314690#comment-17314690
]
Alexey Zotov edited comment on CASSANDRA-16360 at 4/13/21, 2:01 PM:
--------------------------------------------------------------------
I've given a try to two more pure Java-based implementations for _CRC32C_
calculation:
- [Commons Codec
library|https://commons.apache.org/proper/commons-codec/apidocs/org/apache/commons/codec/digest/PureJavaCrc32C.html]
- Custom copy of CRC32C from Java9
Based on the benchmarking, I can see that they provide nearly the same results
as Snappy's implementation (the results are consistent with other runs, run was
made using Java 11.0.9):
{code:java}
[java] Benchmark (bufferSize) Mode Cnt Score
Error Units
[java] ChecksumBench.benchComCdcPureJavaCrc32c 31 avgt 5 95.370 ±
4.438 ns/op
[java] ChecksumBench.benchComCdcPureJavaCrc32c 131 avgt 5 212.066 ±
5.835 ns/op
[java] ChecksumBench.benchComCdcPureJavaCrc32c 517 avgt 5 768.599 ±
12.188 ns/op
[java] ChecksumBench.benchComCdcPureJavaCrc32c 2041 avgt 5 2884.305 ±
86.143 ns/op
[java] ChecksumBench.benchJava9PortForCRC32C 31 avgt 5 89.839 ±
2.027 ns/op
[java] ChecksumBench.benchJava9PortForCRC32C 131 avgt 5 233.196 ±
22.965 ns/op
[java] ChecksumBench.benchJava9PortForCRC32C 517 avgt 5 738.161 ±
17.959 ns/op
[java] ChecksumBench.benchJava9PortForCRC32C 2041 avgt 5 2561.263 ±
13.935 ns/op
[java] ChecksumBench.benchSnappyPureJavaCrc32c 31 avgt 5 96.605 ±
4.376 ns/op
[java] ChecksumBench.benchSnappyPureJavaCrc32c 131 avgt 5 239.617 ±
11.468 ns/op
[java] ChecksumBench.benchSnappyPureJavaCrc32c 517 avgt 5 815.275 ±
8.615 ns/op
[java] ChecksumBench.benchSnappyPureJavaCrc32c 2041 avgt 5 2960.709 ±
68.225 ns/op
{code}
Looks like there is no easy way to improve pure Java implementations for
_CRC32C_ calculation. And we need to decide whether we're ok to move forward
with migration from _CRC32_ to _CRC32C_.
I'm waiting for some input.
----
I've run the same scenarios (obviously except of _benchCrc32c_ and
_benchCrc32cNoIntrinsic_) on Java 1.8.0_281 and results are consistent with
java 11.0.9.
----
The above changes are experimental. I'm not going to push them to the existing
PR, however, they are available for review at:
[https://github.com/alex-ninja/cassandra/pull/1/files].
was (Author: azotcsit):
I've given a try to two more pure Java-based implementations for _CRC32C_
calculation:
- [Common Codec
library|https://commons.apache.org/proper/commons-codec/apidocs/org/apache/commons/codec/digest/PureJavaCrc32C.html]
- Custom copy of CRC32C from Java9
Based on the benchmarking, I can see that they provide nearly the same results
as Snappy's implementation (the results are consistent with other runs, run was
made using Java 11.0.9):
{code:java}
[java] Benchmark (bufferSize) Mode Cnt Score
Error Units
[java] ChecksumBench.benchComCdcPureJavaCrc32c 31 avgt 5 95.370 ±
4.438 ns/op
[java] ChecksumBench.benchComCdcPureJavaCrc32c 131 avgt 5 212.066 ±
5.835 ns/op
[java] ChecksumBench.benchComCdcPureJavaCrc32c 517 avgt 5 768.599 ±
12.188 ns/op
[java] ChecksumBench.benchComCdcPureJavaCrc32c 2041 avgt 5 2884.305 ±
86.143 ns/op
[java] ChecksumBench.benchJava9PortForCRC32C 31 avgt 5 89.839 ±
2.027 ns/op
[java] ChecksumBench.benchJava9PortForCRC32C 131 avgt 5 233.196 ±
22.965 ns/op
[java] ChecksumBench.benchJava9PortForCRC32C 517 avgt 5 738.161 ±
17.959 ns/op
[java] ChecksumBench.benchJava9PortForCRC32C 2041 avgt 5 2561.263 ±
13.935 ns/op
[java] ChecksumBench.benchSnappyPureJavaCrc32c 31 avgt 5 96.605 ±
4.376 ns/op
[java] ChecksumBench.benchSnappyPureJavaCrc32c 131 avgt 5 239.617 ±
11.468 ns/op
[java] ChecksumBench.benchSnappyPureJavaCrc32c 517 avgt 5 815.275 ±
8.615 ns/op
[java] ChecksumBench.benchSnappyPureJavaCrc32c 2041 avgt 5 2960.709 ±
68.225 ns/op
{code}
Looks like there is no easy way to improve pure Java implementations for
_CRC32C_ calculation. And we need to decide whether we're ok to move forward
with migration from _CRC32_ to _CRC32C_.
I'm waiting for some input.
----
I've run the same scenarios (obviously except of _benchCrc32c_ and
_benchCrc32cNoIntrinsic_) on Java 1.8.0_281 and results are consistent with
java 11.0.9.
----
The above changes are experimental. I'm not going to push them to the existing
PR, however, they are available for review at:
https://github.com/alex-ninja/cassandra/pull/1/files.
> CRC32 is inefficient on x86
> ---------------------------
>
> Key: CASSANDRA-16360
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16360
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Client
> Reporter: Avi Kivity
> Priority: Normal
> Labels: protocolv5
> Fix For: 4.0.x
>
>
> The client/server protocol specifies CRC24 and CRC32 as the checksum
> algorithm (cql_protocol_V5_framing.asc). Those however are expensive to
> compute; this affects both the client and the server.
>
> A better checksum algorithm is CRC32C, which has hardware support on x86 (as
> well as other modern architectures).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
