[
https://issues.apache.org/jira/browse/HADOOP-10674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14025912#comment-14025912
]
Tsz Wo Nicholas Sze commented on HADOOP-10674:
----------------------------------------------
Interesting, the result is difference in Java 7. java.util.zip.CRC32 is the
best in Java 7
java.version = 1.7.0_45
java.runtime.name = Java(TM) SE Runtime Environment
java.runtime.version = 1.7.0_45-b18
java.vm.version = 24.45-b08
java.vm.vendor = Oracle Corporation
java.vm.name = Java HotSpot(TM) 64-Bit Server VM
java.vm.specification.version = 1.7
java.specification.version = 1.7
os.arch = x86_64
os.name = Mac OS X
os.version = 10.9.3
Performance Table (The unit is MB/sec)
|| Num Bytes || CRC32 || PureJavaCrc32 | % diff || PureJavaCrc32new | % diff
| % diff ||
| 1 | 10.463 | 145.591 | 1291.5% | 156.814 |
1398.7% | 7.7% |
| 2 | 22.984 | 267.600 | 1064.3% | 286.817 |
1147.9% | 7.2% |
| 4 | 42.437 | 338.265 | 697.1% | 338.419 | 697.5%
| 0.0% |
| 8 | 77.802 | 781.078 | 903.9% | 638.547 | 720.7%
| -18.2% |
| 16 | 146.058 | 1086.145 | 643.6% | 863.587 | 491.3%
| -20.5% |
| 32 | 256.917 | 1301.898 | 406.7% | 1030.223 | 301.0%
| -20.9% |
| 64 | 449.421 | 1455.786 | 223.9% | 1164.038 | 159.0%
| -20.0% |
| 128 | 753.642 | 1557.765 | 106.7% | 1246.300 | 65.4%
| -20.0% |
| 256 | 1307.296 | 1532.365 | 17.2% | 1245.948 | -4.7%
| -18.7% |
| 512 | 1811.799 | 1598.921 | -11.7% | 1492.607 | -17.6%
| -6.6% |
| 1024 | 2279.771 | 1570.552 | -31.1% | 1551.045 | -32.0%
| -1.2% |
| 2048 | 2541.397 | 1639.197 | -35.5% | 1535.380 | -39.6%
| -6.3% |
| 4096 | 2733.641 | 1648.720 | -39.7% | 1622.368 | -40.7%
| -1.6% |
| 8192 | 2806.648 | 1630.906 | -41.9% | 1643.027 | -41.5%
| 0.7% |
| 16384 | 2892.917 | 1629.971 | -43.7% | 1646.938 | -43.1%
| 1.0% |
| 32768 | 2944.763 | 1596.667 | -45.8% | 1661.647 | -43.6%
| 4.1% |
| 65536 | 2961.181 | 1636.034 | -44.8% | 1656.229 | -44.1%
| 1.2% |
| 131072 | 2743.073 | 1627.546 | -40.7% | 1663.785 | -39.3%
| 2.2% |
| 262144 | 2979.100 | 1661.431 | -44.2% | 1659.321 | -44.3%
| -0.1% |
| 524288 | 2966.809 | 1627.857 | -45.1% | 1641.636 | -44.7%
| 0.8% |
| 1048576 | 2987.165 | 1645.096 | -44.9% | 1596.109 | -46.6%
| -3.0% |
| 2097152 | 2876.469 | 1588.168 | -44.8% | 1651.272 | -42.6%
| 4.0% |
| 4194304 | 2884.637 | 1623.130 | -43.7% | 1633.612 | -43.4%
| 0.6% |
| 8388608 | 2956.257 | 1654.131 | -44.0% | 1643.217 | -44.4%
| -0.7% |
| 16777216 | 2951.417 | 1645.519 | -44.2% | 1243.503 | -57.9%
| -24.4% |
> Rewrite the PureJavaCrc32 loop for performance improvement
> ----------------------------------------------------------
>
> Key: HADOOP-10674
> URL: https://issues.apache.org/jira/browse/HADOOP-10674
> Project: Hadoop Common
> Issue Type: Improvement
> Components: performance, util
> Reporter: Tsz Wo Nicholas Sze
> Assignee: Tsz Wo Nicholas Sze
> Attachments: c10674_20140609.patch
>
>
> Below are some performance improvement opportunities performance improvement
> in PureJavaCrc32.
> - eliminate "off += 8; len -= 8;"
> - replace T8_x_start with hard coded constants
> - eliminate c0 - c7 local variables
> In my machine, there are 30% to 50% improvement for most of the cases.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
