[
https://issues.apache.org/jira/browse/HADOOP-10674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsz Wo Nicholas Sze updated HADOOP-10674:
-----------------------------------------
Attachment: c10674_20140609b.patch
c10674_20140609b.patch: bumps up to 40% - 5x% for most cases in Java 6.
java.version = 1.6.0_65
java.runtime.name = Java(TM) SE Runtime Environment
java.runtime.version = 1.6.0_65-b14-462-11M4609
java.vm.version = 20.65-b04-462
java.vm.vendor = Apple Inc.
java.vm.name = Java HotSpot(TM) 64-Bit Server VM
java.vm.specification.version = 1.0
java.specification.version = 1.6
os.arch = x86_64
os.name = Mac OS X
os.version = 10.9.3
Performance Table (The unit is MB/sec)
|| Num Bytes || CRC32 || PureJavaCrc32 | % diff || PureJavaCrc32new | % diff
| % diff ||
| 1 | 16.967 | 174.299 | 927.3% | 173.020 | 919.7%
| -0.7% |
| 2 | 33.743 | 281.089 | 733.0% | 274.722 | 714.2%
| -2.3% |
| 4 | 65.091 | 335.143 | 414.9% | 322.238 | 395.1%
| -3.9% |
| 8 | 114.630 | 634.040 | 453.1% | 671.183 | 485.5%
| 5.9% |
| 16 | 180.051 | 762.082 | 323.3% | 984.305 | 446.7%
| 29.2% |
| 32 | 251.243 | 937.990 | 273.3% | 1245.076 | 395.6%
| 32.7% |
| 64 | 312.430 | 1047.189 | 235.2% | 1435.815 | 359.6%
| 37.1% |
| 128 | 358.860 | 1034.427 | 188.3% | 1584.786 | 341.6%
| 53.2% |
| 256 | 385.658 | 1132.062 | 193.5% | 1603.889 | 315.9%
| 41.7% |
| 512 | 399.778 | 1131.882 | 183.1% | 1656.529 | 314.4%
| 46.4% |
| 1024 | 409.220 | 1126.007 | 175.2% | 1673.662 | 309.0%
| 48.6% |
| 2048 | 409.483 | 1129.983 | 176.0% | 1738.233 | 324.5%
| 53.8% |
| 4096 | 412.962 | 1157.073 | 180.2% | 1715.749 | 315.5%
| 48.3% |
| 8192 | 410.371 | 1101.663 | 168.5% | 1739.911 | 324.0%
| 57.9% |
| 16384 | 417.652 | 1121.469 | 168.5% | 1751.649 | 319.4%
| 56.2% |
| 32768 | 410.410 | 1107.276 | 169.8% | 1737.289 | 323.3%
| 56.9% |
| 65536 | 405.983 | 1185.405 | 192.0% | 1710.018 | 321.2%
| 44.3% |
| 131072 | 417.611 | 1170.179 | 180.2% | 1752.896 | 319.7%
| 49.8% |
| 262144 | 412.452 | 1131.202 | 174.3% | 1728.258 | 319.0%
| 52.8% |
| 524288 | 418.033 | 1190.620 | 184.8% | 1757.276 | 320.4%
| 47.6% |
| 1048576 | 409.731 | 1163.266 | 183.9% | 1762.648 | 330.2%
| 51.5% |
| 2097152 | 419.334 | 1123.300 | 167.9% | 1724.393 | 311.2%
| 53.5% |
| 4194304 | 415.636 | 1133.727 | 172.8% | 1735.664 | 317.6%
| 53.1% |
| 8388608 | 419.852 | 1114.788 | 165.5% | 1731.695 | 312.5%
| 55.3% |
| 16777216 | 413.675 | 1134.631 | 174.3% | 1702.875 | 311.6%
| 50.1% |
> Rewrite the PureJavaCrc32 loop for performance improvement
> ----------------------------------------------------------
>
> Key: HADOOP-10674
> URL: https://issues.apache.org/jira/browse/HADOOP-10674
> Project: Hadoop Common
> Issue Type: Improvement
> Components: performance, util
> Reporter: Tsz Wo Nicholas Sze
> Assignee: Tsz Wo Nicholas Sze
> Attachments: c10674_20140609.patch, c10674_20140609b.patch
>
>
> Below are some performance improvement opportunities performance improvement
> in PureJavaCrc32.
> - eliminate "off += 8; len -= 8;"
> - replace T8_x_start with hard coded constants
> - eliminate c0 - c7 local variables
> In my machine, there are 30% to 50% improvement for most of the cases.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
