[
https://issues.apache.org/jira/browse/HADOOP-6166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsz Wo (Nicholas), SZE updated HADOOP-6166:
-------------------------------------------
Attachment: c6166_20090728.patch
c6166_20090728.patch: included Crc32_16_16
> For the 32 bit results, try passing -server on the command line. ...
Here is the result:
java.version = 1.6.0_14
java.runtime.name = Java(TM) SE Runtime Environment
java.runtime.version = 1.6.0_14-b08
java.vm.version = 14.0-b16
java.vm.vendor = Sun Microsystems Inc.
java.vm.name = Java HotSpot(TM) Server VM
java.vm.specification.version = 1.0
java.specification.version = 1.6
os.arch = x86
os.name = Windows XP
os.version = 5.1
||num bytes||PureJavaCrc32 MB/sec||PureJavaCrc32New MB/sec||Crc32_3_2
MB/sec||Crc32_4_3 MB/sec||Crc32_5_5 MB/sec||Crc32_6_6 MB/sec||Crc32_8_8
MB/sec||Crc32_12_12 MB/sec||Crc32_16_16 MB/sec||
| 8 | 138.935 | 148.510 | 133.888 | 174.420 | 142.309 | 148.559 |
202.270 | 125.889 | 117.607 |
| 16 | 195.238 | 179.688 | 194.082 | 196.024 | 202.448 | 174.408 |
231.516 | 181.476 | 249.847 |
| 32 | 239.042 | 212.647 | 218.873 | 214.975 | 238.313 | 234.569 |
285.546 | 222.713 | 282.443 |
| 64 | 267.240 | 236.977 | 248.711 | 224.998 | 272.373 | 259.976 |
314.990 | 268.683 | 306.000 |
| 128 | 282.564 | 261.874 | 258.325 | 195.558 | 183.524 | 290.901 |
339.453 | 307.891 | 285.557 |
| 256 | 286.647 | 271.146 | 270.484 | 224.961 | 288.691 | 307.519 |
352.148 | 337.360 | 312.192 |
| 512 | 298.539 | 276.192 | 274.773 | 236.895 | 336.279 | 315.217 |
361.232 | 346.809 | 319.615 |
| 1024 | 303.658 | 279.882 | 276.542 | 236.183 | 340.919 | 325.135 |
364.909 | 352.689 | 319.080 |
| 2048 | 309.358 | 285.787 | 273.328 | 236.416 | 345.868 | 327.777 |
368.106 | 357.019 | 321.033 |
| 4096 | 306.306 | 285.192 | 272.680 | 237.541 | 343.045 | 327.025 |
368.837 | 358.270 | 322.088 |
| 8192 | 307.772 | 288.171 | 272.977 | 237.316 | 348.833 | 328.908 |
373.525 | 361.827 | 322.454 |
| 16384 | 307.900 | 286.654 | 273.482 | 236.011 | 332.936 | 328.303 |
370.397 | 359.706 | 320.460 |
| 32768 | 302.599 | 285.929 | 273.000 | 237.496 | 343.129 | 328.161 |
368.144 | 360.141 | 320.854 |
| 65536 | 305.564 | 285.796 | 273.027 | 236.645 | 342.567 | 329.054 |
369.318 | 360.611 | 322.333 |
| 131072 | 306.763 | 285.466 | 274.336 | 237.648 | 344.286 | 329.910 |
373.027 | 360.100 | 320.236 |
| 262144 | 302.322 | 286.444 | 273.267 | 236.971 | 345.512 | 327.882 |
370.549 | 358.936 | 320.964 |
| 524288 | 304.555 | 284.659 | 272.150 | 235.174 | 342.026 | 327.074 |
369.213 | 359.436 | 316.547 |
| 1048576 | 301.722 | 279.686 | 271.529 | 235.130 | 338.665 | 324.743 |
365.818 | 352.513 | 315.451 |
| 2097152 | 301.360 | 282.853 | 270.846 | 232.843 | 336.175 | 322.065 |
362.790 | 356.372 | 317.965 |
| 4194304 | 298.921 | 283.021 | 269.376 | 233.498 | 336.376 | 321.957 |
365.402 | 354.546 | 299.699 |
| 8388608 | 250.164 | 281.916 | 269.071 | 234.353 | 338.636 | 325.124 |
365.995 | 353.549 | 312.460 |
| 16777216 | 290.762 | 264.850 | 270.366 | 235.145 | 338.756 | 321.101 |
364.583 | 353.767 | 316.974 |
> Are you specifying a -Xmx memory value? What about -Xms?
I have -Xmx512m but no -Xms. Any suggestion?
> It is also curious that the PureJavaCrc32New - which only changes the loop
> style ...
This trick does not always work: PureJavaCrc32New was slower in the results
shown above.
> I think there may be some improvement possible in the 8_8 case in how the 9
> XORs at the end are done. ...
Yeah, we should try.
Thanks, Scott.
> Improve PureJavaCrc32
> ---------------------
>
> Key: HADOOP-6166
> URL: https://issues.apache.org/jira/browse/HADOOP-6166
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: c6166_20090722.patch, c6166_20090722_benchmark_32VM.txt,
> c6166_20090722_benchmark_64VM.txt, c6166_20090727.patch, c6166_20090728.patch
>
>
> Got some ideas to improve CRC32 calculation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
