[
https://issues.apache.org/jira/browse/HADOOP-6166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12734762#action_12734762
]
Tsz Wo (Nicholas), SZE commented on HADOOP-6166:
------------------------------------------------
Unfortunately, Crc32_4_3 only wins on a 32-bit vm over
TestPureJavaCrc32.PerformanceTest but not 64-bit vm.
- 32-bit vm
||num bytes||CRC32 MB/sec||Crc32_4_2 MB/sec||Crc32_4_3 MB/sec||Crc32_3_2
MB/sec||PureJavaCrc32 MB/sec||
| 1 |4.504 |52.409 |55.779 |46.603 |59.146
|
| 2 |8.825 |86.590 |87.938 |77.267 |89.942
|
| 4 |17.254 |119.824 |151.929 |120.808
|146.983 |
| 8 |32.037 |147.222 |202.527 |161.984
|174.844 |
| 16 |59.078 |161.879 |231.467 |195.635
|228.018 |
| 32 |100.267 |176.767 |276.844 |241.295
|244.502 |
| 64 |148.985 |178.250 |283.511 |269.368
|263.209 |
| 128 |199.763 |185.639 |294.116 |271.943
|259.021 |
| 256 |232.751 |179.525 |290.357 |259.453
|256.891 |
| 512 |255.430 |178.217 |296.907 |280.763
|257.362 |
| 1024 |262.274 |172.033 |289.806 |277.863
|261.793 |
| 2048 |273.744 |187.468 |299.271 |286.387
|272.611 |
| 4096 |289.373 |186.306 |293.845 |276.021
|266.067 |
| 8192 |290.282 |184.625 |296.723 |285.097
|271.503 |
| 16384 |298.959 |180.863 |291.081 |250.583
|199.536 |
| 32768 |277.718 |184.078 |293.156 |285.722
|270.377 |
| 65536 |300.016 |186.439 |298.946 |283.990
|271.268 |
| 131072 |298.971 |186.754 |298.417 |283.949
|268.240 |
| 262144 |299.688 |184.124 |296.014 |281.633
|265.799 |
| 524288 |282.488 |176.217 |288.120 |284.030
|267.917 |
| 1048576 |294.852 |185.167 |291.499 |279.362
|267.438 |
| 2097152 |296.117 |174.667 |281.145 |272.180
|260.837 |
| 4194304 |283.934 |173.777 |279.931 |271.393
|259.805 |
| 8388608 |289.455 |177.829 |291.535 |269.850
|259.513 |
| 16777216 |284.204 |177.449 |290.489 |276.586
|265.657 |
- 64-bit vm
||num bytes||CRC32 MB/sec||Crc32_4_2 MB/sec||Crc32_4_3 MB/sec||Crc32_3_2
MB/sec||PureJavaCrc32 MB/sec||
| 1 |7.636 |80.107 |99.658 |77.283 |34.446
|
| 2 |14.598 |116.202 |110.091 |94.056
|106.498 |
| 4 |27.786 |152.932 |197.294 |147.532
|175.766 |
| 8 |50.159 |153.598 |194.617 |163.596
|197.350 |
| 16 |85.036 |177.761 |258.683 |237.917
|278.764 |
| 32 |130.278 |180.486 |310.024 |281.343
|342.374 |
| 64 |177.501 |181.663 |343.592 |320.385
|384.938 |
| 128 |217.128 |181.836 |366.965 |338.893
|411.724 |
| 256 |245.690 |182.637 |379.003 |348.981
|425.874 |
| 512 |262.085 |181.103 |381.961 |355.506
|428.103 |
| 1024 |271.307 |179.753 |381.658 |356.488
|433.800 |
| 2048 |276.640 |180.451 |378.667 |351.067
|437.275 |
| 4096 |278.435 |179.881 |372.762 |347.728
|437.209 |
| 8192 |279.883 |180.776 |377.241 |351.178
|439.571 |
| 16384 |281.385 |180.775 |377.493 |353.361
|439.606 |
| 32768 |281.434 |180.703 |378.047 |353.656
|438.703 |
| 65536 |281.354 |180.914 |377.805 |353.130
|437.152 |
| 131072 |280.941 |180.288 |377.164 |353.340
|438.806 |
| 262144 |282.056 |180.910 |378.514 |354.320
|438.208 |
| 524288 |281.066 |180.177 |377.148 |352.832
|437.183 |
| 1048576 |281.668 |180.790 |378.412 |354.059
|438.755 |
| 2097152 |282.162 |180.545 |377.918 |353.841
|438.497 |
| 4194304 |281.379 |179.018 |376.287 |352.240
|436.963 |
| 8388608 |279.929 |178.058 |371.618 |349.405
|430.993 |
| 16777216 |278.974 |177.577 |371.729 |347.971
|429.326 |
> Improve PureJavaCrc32
> ---------------------
>
> Key: HADOOP-6166
> URL: https://issues.apache.org/jira/browse/HADOOP-6166
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: c6166_20090722.patch, c6166_20090722_benchmark_32VM.txt,
> c6166_20090722_benchmark_64VM.txt
>
>
> Got some ideas to improve CRC32 calculation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
