[
https://issues.apache.org/jira/browse/HADOOP-10674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14025847#comment-14025847
]
Tsz Wo Nicholas Sze commented on HADOOP-10674:
----------------------------------------------
java.version = 1.6.0_65
java.runtime.name = Java(TM) SE Runtime Environment
java.runtime.version = 1.6.0_65-b14-462-11M4609
java.vm.version = 20.65-b04-462
java.vm.vendor = Apple Inc.
java.vm.name = Java HotSpot(TM) 64-Bit Server VM
java.vm.specification.version = 1.0
java.specification.version = 1.6
os.arch = x86_64
os.name = Mac OS X
os.version = 10.9.3
Performance Table (The unit is MB/sec)
|| Num Bytes || CRC32 || PureJavaCrc32 | % diff || PureJavaCrc32new | % diff
||
| 1 | 17.170 | 174.807 | 918.1% | 184.712 | 5.7%
|
| 2 | 35.428 | 290.235 | 719.2% | 275.524 | -5.1%
|
| 4 | 67.132 | 338.614 | 404.4% | 334.880 | -1.1%
|
| 8 | 115.325 | 623.372 | 440.5% | 658.562 | 5.6%
|
| 16 | 180.665 | 829.639 | 359.2% | 926.114 | 11.6%
|
| 32 | 254.776 | 977.622 | 283.7% | 1186.548 | 21.4%
|
| 64 | 315.420 | 1072.557 | 240.0% | 1292.799 | 20.5%
|
| 128 | 360.204 | 1111.217 | 208.5% | 1483.043 | 33.5%
|
| 256 | 390.422 | 1139.855 | 192.0% | 1557.234 | 36.6%
|
| 512 | 404.637 | 1003.717 | 148.1% | 1590.339 | 58.4%
|
| 1024 | 413.142 | 1167.351 | 182.6% | 1588.641 | 36.1%
|
| 2048 | 413.260 | 1128.091 | 173.0% | 1630.324 | 44.5%
|
| 4096 | 412.476 | 1145.957 | 177.8% | 1660.634 | 44.9%
|
| 8192 | 416.169 | 1185.569 | 184.9% | 1600.820 | 35.0%
|
| 16384 | 420.509 | 1181.968 | 181.1% | 1601.702 | 35.5%
|
| 32768 | 418.372 | 1141.980 | 173.0% | 1610.671 | 41.0%
|
| 65536 | 418.190 | 1146.984 | 174.3% | 1682.749 | 46.7%
|
| 131072 | 415.032 | 1180.082 | 184.3% | 1571.806 | 33.2%
|
| 262144 | 414.864 | 1186.768 | 186.1% | 1684.632 | 42.0%
|
| 524288 | 421.510 | 1196.519 | 183.9% | 1580.774 | 32.1%
|
| 1048576 | 422.179 | 1187.153 | 181.2% | 1628.685 | 37.2%
|
| 2097152 | 422.650 | 1108.484 | 162.3% | 1642.879 | 48.2%
|
| 4194304 | 418.723 | 1114.216 | 166.1% | 1663.223 | 49.3%
|
| 8388608 | 419.063 | 1034.310 | 146.8% | 1631.987 | 57.8%
|
| 16777216 | 421.176 | 1181.302 | 180.5% | 1636.996 | 38.6%
|
> Rewrite the PureJavaCrc32 loop for performance improvement
> ----------------------------------------------------------
>
> Key: HADOOP-10674
> URL: https://issues.apache.org/jira/browse/HADOOP-10674
> Project: Hadoop Common
> Issue Type: Bug
> Components: performance, util
> Reporter: Tsz Wo Nicholas Sze
> Assignee: Tsz Wo Nicholas Sze
>
> Below are some performance improvement opportunities performance improvement
> in PureJavaCrc32.
> - eliminate "off += 8; len -= 8;"
> - replace T8_x_start with hard coded constants
> - eliminate c0 - c7 local variables
> In my machine, there are 30% to 50% improvement for most of the cases.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
