[
https://issues.apache.org/jira/browse/CASSANDRA-3610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174952#comment-13174952
]
Vijay commented on CASSANDRA-3610:
----------------------------------
i modified the test class from HADOOP-6148 (bigger is better)
the result is:
||bytes||PureJava MB/sec||Native MB/sec||Random PureJava MB/sec||Native MB/sec||
| 1 |40.525 |12.118 |
| 2 |151.706 |23.046 |
| 4 |202.082 |46.778 |
| 8 |309.846 |88.030 |
| 16 |330.308 |152.373 |
| 32 |384.488 |247.181 |
| 64 |413.354 |350.407 || PureJava |0-64 |260.692
MB/sec|| Native |0-64 |198.142 MB/sec|
| 128 |429.670 |450.514 || PureJava |0-128 |323.232
MB/sec|| Native |0-128 |299.766 MB/sec|
| 256 |437.959 |521.985 || PureJava |0-256 |376.471
MB/sec|| Native |0-256 |405.063 MB/sec|
| 512 |439.036 |564.909 || PureJava |0-512 |410.257
MB/sec|| Native |0-512 |490.422 MB/sec|
| 1024 |441.101 |587.137 || PureJava |0-1024 |426.669
MB/sec|| Native |0-1024 |549.357 MB/sec|
| 2048 |443.893 |601.072 || PureJava |0-2048 |436.865
MB/sec|| Native |0-2048 |579.190 MB/sec|
| 4096 |444.964 |608.434 || PureJava |0-4096 |441.389
MB/sec|| Native |0-4096 |600.953 MB/sec|
| 8192 |445.737 |610.422 || PureJava |0-8192 |444.453
MB/sec|| Native |0-8192 |609.527 MB/sec|
| 16384 |447.157 |615.396 || PureJava |0-16384
|447.575 MB/sec|| Native |0-16384 |615.407 MB/sec|
| 32768 |446.494 |615.482 || PureJava |0-32768
|446.071 MB/sec|| Native |0-32768 |615.393 MB/sec|
| 65536 |446.550 |616.318 || PureJava |0-65536
|446.011 MB/sec|| Native |0-65536 |615.462 MB/sec|
| 131072 |446.603 |616.220 || PureJava |0-131072
|446.256 MB/sec|| Native |0-131072 |615.432 MB/sec|
| 262144 |445.461 |613.127 || PureJava |0-262144
|446.049 MB/sec|| Native |0-262144 |616.090 MB/sec|
| 524288 |445.208 |612.968 || PureJava |0-524288
|445.257 MB/sec|| Native |0-524288 |610.429 MB/sec|
| 1048576 |438.190 |607.029 || PureJava |0-1048576
|434.039 MB/sec|| Native |0-1048576 |603.446 MB/sec|
| 2097152 |428.816 |603.694 || PureJava |0-2097152
|423.804 MB/sec|| Native |0-2097152 |607.624 MB/sec|
| 4194304 |421.485 |606.262 || PureJava |0-4194304
|439.061 MB/sec|| Native |0-4194304 |609.371 MB/sec|
| 8388608 |424.566 |598.640 || PureJava |0-8388608
|426.544 MB/sec|| Native |0-8388608 |608.484 MB/sec|
| 16777216 |433.218 |602.733 || PureJava |0-16777216
|434.810 MB/sec|| Native |0-16777216 |601.179 MB/sec|
> Checksum improvement for CompressedRandomAccessReader
> -----------------------------------------------------
>
> Key: CASSANDRA-3610
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3610
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 1.1
> Environment: JVM
> Reporter: Vijay
> Assignee: Vijay
> Priority: Minor
> Fix For: 1.1
>
> Attachments: 0001-use-pure-java-CRC32-v2.patch,
> 0001-use-pure-java-CRC32.patch
>
>
> When compression is on, Currently we see checksum taking up about 40% of the
> CPU more than snappy library.
> Looks like hadoop solved it by implementing their own checksum, we can either
> use it or implement something like that.
> http://images.slidesharecdn.com/1toddlipconyanpeichen-cloudera-hadoopandperformance-final-111110132228-phpapp01-slide-15-768.jpg?1321043717
> in our test env it provided 50% improvement over native implementation which
> uses jni to call the OS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
