[
https://issues.apache.org/jira/browse/HADOOP-6166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsz Wo (Nicholas), SZE updated HADOOP-6166:
-------------------------------------------
Attachment: c6166_20090727.patch
Not yet able to improve PureJavaCrc32 in my 64-bit machine but had a lot of fun
last weekends.
c6166_20090727.patch: moved the codes to common (finally). Please try it when
you have time.
- 64-bit
java.version = 1.6.0_10
java.runtime.name = Java(TM) SE Runtime Environment
java.runtime.version = 1.6.0_10-b33
java.vm.version = 11.0-b15
java.vm.vendor = Sun Microsystems Inc.
java.vm.name = Java HotSpot(TM) 64-Bit Server VM
java.vm.specification.version = 1.0
java.specification.version = 1.6
os.arch = amd64
os.name = Linux
os.version = 2.6.9-55.ELsmp
||num bytes||PureJavaCrc32 MB/sec||PureJavaCrc32New MB/sec||Crc32_3_2
MB/sec||Crc32_4_3 MB/sec||Crc32_5_5 MB/sec||Crc32_6_6 MB/sec||Crc32_8_8
MB/sec||Crc32_12_12 MB/sec||
| 8 | 157.986 | 102.628 | 135.926 | 204.207 | 218.584 | 239.862 |
253.886 | 213.072 |
| 16 | 245.381 | 214.363 | 238.798 | 202.284 | 261.246 | 207.342 |
219.935 | 245.648 |
| 32 | 331.296 | 290.766 | 283.689 | 218.582 | 310.499 | 329.800 |
283.757 | 273.074 |
| 64 | 405.822 | 345.573 | 325.067 | 224.623 | 344.732 | 345.010 |
311.538 | 346.970 |
| 128 | 451.240 | 378.875 | 343.498 | 226.853 | 391.824 | 392.462 |
323.504 | 391.573 |
| 256 | 479.728 | 410.574 | 352.432 | 226.939 | 416.448 | 396.344 |
331.537 | 415.233 |
| 512 | 488.917 | 425.214 | 355.640 | 227.120 | 424.109 | 409.781 |
335.965 | 427.655 |
| 1024 | 499.820 | 433.135 | 358.441 | 225.953 | 430.212 | 414.286 |
337.652 | 431.440 |
| 2048 | 504.199 | 438.373 | 352.913 | 223.754 | 435.921 | 417.888 |
339.190 | 438.454 |
| 4096 | 509.100 | 441.553 | 351.305 | 222.355 | 438.667 | 420.657 |
341.057 | 441.063 |
| 8192 | 511.632 | 439.242 | 352.058 | 222.469 | 439.568 | 422.009 |
341.427 | 447.972 |
| 16384 | 510.829 | 444.631 | 354.097 | 222.488 | 439.707 | 419.661 |
341.200 | 451.286 |
| 32768 | 507.353 | 437.758 | 354.601 | 222.503 | 436.775 | 416.266 |
339.704 | 449.830 |
| 65536 | 507.335 | 434.042 | 354.837 | 222.682 | 436.742 | 417.825 |
339.761 | 449.868 |
| 131072 | 507.473 | 431.449 | 355.014 | 222.748 | 436.477 | 417.910 |
339.835 | 449.958 |
| 262144 | 507.548 | 429.451 | 354.932 | 222.632 | 436.698 | 417.783 |
339.852 | 449.936 |
| 524288 | 507.322 | 428.618 | 355.142 | 222.491 | 436.584 | 417.715 |
339.826 | 450.146 |
| 1048576 | 507.148 | 428.778 | 354.769 | 222.534 | 436.506 | 417.819 |
339.830 | 450.032 |
| 2097152 | 506.610 | 432.981 | 354.596 | 222.261 | 436.080 | 417.573 |
339.933 | 449.623 |
| 4194304 | 504.503 | 432.501 | 352.918 | 221.669 | 432.956 | 414.626 |
337.668 | 445.489 |
| 8388608 | 498.208 | 428.943 | 348.488 | 219.868 | 429.455 | 411.497 |
335.448 | 440.899 |
| 16777216 | 497.184 | 423.245 | 348.105 | 219.657 | 427.288 | 410.788 |
334.992 | 440.603 |
- 32-bit
java.version = 1.6.0_14
java.runtime.name = Java(TM) SE Runtime Environment
java.runtime.version = 1.6.0_14-b08
java.vm.version = 14.0-b16
java.vm.vendor = Sun Microsystems Inc.
java.vm.name = Java HotSpot(TM) Client VM
java.vm.specification.version = 1.0
java.specification.version = 1.6
os.arch = x86
os.name = Windows XP
os.version = 5.1
||num bytes||PureJavaCrc32 MB/sec||PureJavaCrc32New MB/sec||Crc32_3_2
MB/sec||Crc32_4_3 MB/sec||Crc32_5_5 MB/sec||Crc32_6_6 MB/sec||Crc32_8_8
MB/sec||Crc32_12_12 MB/sec||
| 8 | 192.776 | 167.821 | 174.386 | 207.658 | 184.504 | 196.480 |
222.191 | 155.052 |
| 16 | 227.810 | 212.454 | 224.370 | 250.812 | 236.371 | 229.370 |
267.719 | 248.392 |
| 32 | 250.230 | 239.881 | 251.688 | 280.112 | 257.619 | 270.357 |
298.951 | 268.026 |
| 64 | 263.695 | 257.785 | 275.683 | 296.554 | 269.474 | 284.615 |
316.642 | 319.204 |
| 128 | 270.873 | 266.500 | 286.942 | 305.744 | 282.172 | 298.166 |
325.899 | 325.806 |
| 256 | 282.205 | 270.751 | 294.155 | 306.224 | 288.460 | 303.445 |
330.728 | 342.959 |
| 512 | 282.529 | 270.063 | 295.103 | 309.134 | 290.043 | 300.008 |
332.325 | 343.466 |
| 1024 | 279.710 | 273.680 | 298.489 | 308.905 | 289.993 | 305.850 |
331.706 | 347.761 |
| 2048 | 279.753 | 274.073 | 293.911 | 304.972 | 285.538 | 308.030 |
334.459 | 350.296 |
| 4096 | 278.830 | 275.688 | 290.520 | 302.634 | 293.201 | 308.455 |
334.498 | 351.761 |
| 8192 | 279.518 | 274.829 | 289.088 | 299.036 | 292.383 | 305.940 |
333.512 | 352.584 |
| 16384 | 278.609 | 251.000 | 287.964 | 303.862 | 293.782 | 308.534 |
333.397 | 347.889 |
| 32768 | 276.124 | 272.805 | 290.125 | 300.458 | 289.993 | 306.447 |
334.985 | 350.833 |
| 65536 | 274.212 | 273.606 | 288.872 | 303.673 | 286.457 | 307.196 |
332.591 | 349.273 |
| 131072 | 275.371 | 272.257 | 289.985 | 303.490 | 289.126 | 303.128 |
330.630 | 349.575 |
| 262144 | 275.607 | 273.878 | 288.080 | 302.439 | 285.862 | 304.965 |
330.775 | 347.044 |
| 524288 | 274.578 | 270.549 | 286.745 | 299.832 | 287.063 | 304.299 |
332.160 | 346.131 |
| 1048576 | 270.002 | 272.333 | 285.866 | 299.702 | 284.005 | 304.845 |
329.455 | 343.377 |
| 2097152 | 268.254 | 265.650 | 285.905 | 297.428 | 286.168 | 302.749 |
329.962 | 344.515 |
| 4194304 | 272.093 | 268.692 | 285.552 | 299.619 | 285.262 | 299.311 |
327.311 | 338.847 |
| 8388608 | 268.156 | 265.971 | 283.967 | 291.225 | 282.974 | 301.162 |
323.698 | 343.482 |
| 16777216 | 271.428 | 270.893 | 285.939 | 299.171 | 284.946 | 302.520 |
328.465 | 343.694 |
> Improve PureJavaCrc32
> ---------------------
>
> Key: HADOOP-6166
> URL: https://issues.apache.org/jira/browse/HADOOP-6166
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: c6166_20090722.patch, c6166_20090722_benchmark_32VM.txt,
> c6166_20090722_benchmark_64VM.txt, c6166_20090727.patch
>
>
> Got some ideas to improve CRC32 calculation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
