[
https://issues.apache.org/jira/browse/CASSANDRA-3610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175035#comment-13175035
]
Vijay commented on CASSANDRA-3610:
----------------------------------
Ooops pasted the wrong data.... the above data is without any Heap settings
hence GC becomes a bottleneck... Plz see the below :)
/usr/java/latest/jre/bin/java -XX:+UseThreadPriorities
-XX:ThreadPriorityPolicy=42 -Xms8G -Xmx8G -Xmn2G
-XX:+HeapDumpOnOutOfMemoryError -Xss128k -XX:+UseParNewGC
-XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8
-XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75 -jar
TestCRC32Performance.jar
||bytes||PureJava MB/sec||Native MB/sec||Random PureJava MB/sec||Native MB/sec||
| 1 |121.124 |11.866 |
| 2 |161.981 |23.851 |
| 4 |204.718 |45.486 |
| 8 |297.229 |76.296 |
| 16 |379.268 |117.326 |
| 32 |440.153 |157.711 |
| 64 |468.143 |193.304 || PureJava |0-64 |272.921
MB/sec|| Native |0-64 |145.289 MB/sec|
| 128 |500.006 |219.657 || PureJava |0-128 |367.816
MB/sec|| Native |0-128 |186.861 MB/sec|
| 256 |511.572 |234.052 || PureJava |0-256 |432.433
MB/sec|| Native |0-256 |214.047 MB/sec|
| 512 |517.550 |242.634 || PureJava |0-512 |474.074
MB/sec|| Native |0-512 |231.047 MB/sec|
| 1024 |516.994 |246.424 || PureJava |0-1024 |498.055
MB/sec|| Native |0-1024 |241.056 MB/sec|
| 2048 |518.095 |248.529 || PureJava |0-2048 |509.960
MB/sec|| Native |0-2048 |245.683 MB/sec|
| 4096 |522.002 |249.755 || PureJava |0-4096 |518.226
MB/sec|| Native |0-4096 |248.062 MB/sec|
| 8192 |522.795 |250.316 || PureJava |0-8192 |520.326
MB/sec|| Native |0-8192 |249.519 MB/sec|
| 16384 |522.521 |250.484 || PureJava |0-16384
|522.480 MB/sec|| Native |0-16384 |250.002 MB/sec|
| 32768 |521.098 |250.604 || PureJava |0-32768
|520.349 MB/sec|| Native |0-32768 |250.494 MB/sec|
| 65536 |520.973 |250.837 || PureJava |0-65536
|520.392 MB/sec|| Native |0-65536 |249.063 MB/sec|
| 131072 |510.129 |248.949 || PureJava |0-131072
|516.246 MB/sec|| Native |0-131072 |249.535 MB/sec|
| 262144 |513.534 |249.506 || PureJava |0-262144
|514.407 MB/sec|| Native |0-262144 |250.617 MB/sec|
| 524288 |519.554 |250.696 || PureJava |0-524288
|520.402 MB/sec|| Native |0-524288 |251.048 MB/sec|
| 1048576 |519.559 |250.557 || PureJava |0-1048576
|520.403 MB/sec|| Native |0-1048576 |250.734 MB/sec|
| 2097152 |519.259 |250.456 || PureJava |0-2097152
|519.337 MB/sec|| Native |0-2097152 |250.299 MB/sec|
| 4194304 |518.649 |250.470 || PureJava |0-4194304
|518.495 MB/sec|| Native |0-4194304 |250.523 MB/sec|
| 8388608 |501.986 |248.044 || PureJava |0-8388608
|509.521 MB/sec|| Native |0-8388608 |248.626 MB/sec|
| 16777216 |508.201 |247.587 || PureJava |0-16777216
|505.258 MB/sec|| Native |0-16777216 |249.558 MB/sec|
> Checksum improvement for CompressedRandomAccessReader
> -----------------------------------------------------
>
> Key: CASSANDRA-3610
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3610
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 1.1
> Environment: JVM
> Reporter: Vijay
> Assignee: Vijay
> Priority: Minor
> Fix For: 1.1
>
> Attachments: 0001-use-pure-java-CRC32-v2.patch,
> 0001-use-pure-java-CRC32.patch
>
>
> When compression is on, Currently we see checksum taking up about 40% of the
> CPU more than snappy library.
> Looks like hadoop solved it by implementing their own checksum, we can either
> use it or implement something like that.
> http://images.slidesharecdn.com/1toddlipconyanpeichen-cloudera-hadoopandperformance-final-111110132228-phpapp01-slide-15-768.jpg?1321043717
> in our test env it provided 50% improvement over native implementation which
> uses jni to call the OS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
