[
https://issues.apache.org/jira/browse/HADOOP-10674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14026142#comment-14026142
]
Rajesh Balamohan commented on HADOOP-10674:
-------------------------------------------
Posting the results for 1.8 (even though it might take lot of time for Hadoop
to move to 1.8).
java.version = 1.8.0_05
java.runtime.name = Java(TM) SE Runtime Environment
java.runtime.version = 1.8.0_05-b13
java.vm.version = 25.5-b02
java.vm.vendor = Oracle Corporation
java.vm.name = Java HotSpot(TM) 64-Bit Server VM
java.vm.specification.version = 1.8
java.specification.version = 1.8
os.arch = x86_64
os.name = Mac OS X
os.version = 10.9.2
Performance Table (The unit is MB/sec)
|| Num Bytes || CRC32 || PureJavaCrc32 | % diff || PureJavaCrc32new | % diff
||
| 1 | 136.199 | 184.501 | 35.5% | 195.550 | 6.0%
|
| 2 | 261.167 | 283.589 | 8.6% | 277.812 | -2.0%
|
| 4 | 316.930 | 325.149 | 2.6% | 323.370 | -0.5%
|
| 8 | 350.402 | 839.801 | 139.7% | 630.399 | -24.9%
|
| 16 | 371.638 | 1191.862 | 220.7% | 845.571 | -29.1%
|
| 32 | 451.213 | 1400.300 | 210.3% | 958.945 | -31.5%
|
| 64 | 810.445 | 1556.288 | 92.0% | 1133.285 | -27.2%
|
| 128 | 1366.747 | 1581.594 | 15.7% | 1189.381 | -24.8%
|
| 256 | 2526.898 | 1497.479 | -40.7% | 1292.694 | -13.7%
|
| 512 | 4407.562 | 1591.387 | -63.9% | 1485.212 | -6.7%
|
| 1024 | 6661.116 | 1633.674 | -75.5% | 1576.161 | -3.5%
|
| 2048 | 8702.155 | 1639.995 | -81.2% | 1599.360 | -2.5%
|
| 4096 | 10258.054 | 1609.719 | -84.3% | 1652.210 | 2.6%
|
| 8192 | 11365.654 | 1655.651 | -85.4% | 1646.133 | -0.6%
|
| 16384 | 12126.954 | 1592.555 | -86.9% | 1679.129 | 5.4%
|
| 32768 | 12155.745 | 1654.580 | -86.4% | 1655.308 | 0.0%
|
| 65536 | 12411.519 | 1641.699 | -86.8% | 1665.409 | 1.4%
|
| 131072 | 12425.978 | 1652.551 | -86.7% | 1568.704 | -5.1%
|
| 262144 | 11402.102 | 1600.620 | -86.0% | 1670.975 | 4.4%
|
| 524288 | 11499.416 | 1618.737 | -85.9% | 1656.208 | 2.3%
|
| 1048576 | 12458.633 | 1639.512 | -86.8% | 1411.573 | -13.9%
|
| 2097152 | 12368.345 | 1552.908 | -87.4% | 1634.488 | 5.3%
|
| 4194304 | 11476.733 | 1581.028 | -86.2% | 1642.247 | 3.9%
|
| 8388608 | 11798.322 | 1543.582 | -86.9% | 1634.050 | 5.9%
|
| 16777216 | 11103.400 | 1641.573 | -85.2% | 1580.520 | -3.7%
|
With 1.7.0_51
java.version = 1.7.0_51
java.runtime.name = Java(TM) SE Runtime Environment
java.runtime.version = 1.7.0_51-b13
java.vm.version = 24.51-b03
java.vm.vendor = Oracle Corporation
java.vm.name = Java HotSpot(TM) 64-Bit Server VM
java.vm.specification.version = 1.7
java.specification.version = 1.7
os.arch = x86_64
os.name = Mac OS X
os.version = 10.9.2
Performance Table (The unit is MB/sec)
|| Num Bytes || CRC32 || PureJavaCrc32 | % diff || PureJavaCrc32new | % diff
||
| 1 | 10.496 | 150.871 | 1337.4% | 182.493 |
21.0% |
| 2 | 21.612 | 284.191 | 1215.0% | 283.723 |
-0.2% |
| 4 | 43.344 | 323.760 | 646.9% | 318.677 | -1.6%
|
| 8 | 79.073 | 818.739 | 935.4% | 632.961 | -22.7%
|
| 16 | 145.119 | 1186.438 | 717.6% | 837.318 | -29.4%
|
| 32 | 255.357 | 1403.678 | 449.7% | 1009.559 | -28.1%
|
| 64 | 499.140 | 1507.390 | 202.0% | 1128.529 | -25.1%
|
| 128 | 890.676 | 1596.368 | 79.2% | 1185.855 | -25.7%
|
| 256 | 1623.995 | 1528.194 | -5.9% | 1258.926 | -17.6%
|
| 512 | 3097.399 | 1591.901 | -48.6% | 1449.768 | -8.9%
|
| 1024 | 4218.714 | 1585.198 | -62.4% | 1588.188 | 0.2%
|
| 2048 | 7154.835 | 1637.164 | -77.1% | 1623.995 | -0.8%
|
| 4096 | 9198.045 | 1639.722 | -82.2% | 1450.458 | -11.5%
|
| 8192 | 10574.143 | 1629.141 | -84.6% | 1682.528 | 3.3%
|
| 16384 | 11368.683 | 1591.268 | -86.0% | 1670.931 | 5.0%
|
| 32768 | 11695.906 | 1625.149 | -86.1% | 1615.529 | -0.6%
|
| 65536 | 11958.146 | 1620.048 | -86.5% | 1676.622 | 3.5%
|
| 131072 | 12567.501 | 1649.230 | -86.9% | 1599.780 | -3.0%
|
| 262144 | 12415.131 | 1583.394 | -87.2% | 1698.063 | 7.2%
|
| 524288 | 12389.894 | 1652.252 | -86.7% | 1685.808 | 2.0%
|
| 1048576 | 11980.532 | 1658.224 | -86.2% | 1673.268 | 0.9%
|
| 2097152 | 11981.653 | 1655.779 | -86.2% | 1682.860 | 1.6%
|
| 4194304 | 12092.584 | 1647.340 | -86.4% | 1628.892 | -1.1%
|
| 8388608 | 12014.267 | 1545.241 | -87.1% | 1621.628 | 4.9%
|
| 16777216 | 10965.476 | 1646.705 | -85.0% | 1606.345 | -2.5%
|
> Rewrite the PureJavaCrc32 loop for performance improvement
> ----------------------------------------------------------
>
> Key: HADOOP-10674
> URL: https://issues.apache.org/jira/browse/HADOOP-10674
> Project: Hadoop Common
> Issue Type: Improvement
> Components: performance, util
> Reporter: Tsz Wo Nicholas Sze
> Assignee: Tsz Wo Nicholas Sze
> Attachments: c10674_20140609.patch, c10674_20140609b.patch
>
>
> Below are some performance improvement opportunities performance improvement
> in PureJavaCrc32.
> - eliminate "off += 8; len -= 8;"
> - replace T8_x_start with hard coded constants
> - eliminate c0 - c7 local variables
> In my machine, there are 30% to 50% improvement for most of the cases.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
