[
https://issues.apache.org/jira/browse/HADOOP-7657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169970#comment-13169970
]
Binglin Chang commented on HADOOP-7657:
---------------------------------------
I have integrated lz4 in native task optimizition(MAPREDUCE-2841), and it did
show better results compare to snappy. Here is some test results for terasort
input data:
[ RUN ] Perf.Compressions
11/12/15 13:04:24 INFO Generate data time: 2.12431s
11/12/15 13:04:26 INFO memset buffer to prevent missing page time: 2.08189s
11/12/15 13:04:26 INFO org.apache.hadoop.io.compress.SnappyCodec
11/12/15 13:04:26 INFO snappy compress origin/compressed time: 0.16029s size:
100.000M/21.944M speed: 623.86M/136.90M
11/12/15 13:04:26 INFO snappy decompress orig/uncompressed time: 0.05860s size:
21.944M/100.000M speed: 374.49M/1706.57M
11/12/15 13:04:26 INFO ratio: 0.219
11/12/15 13:04:26 INFO org.apache.hadoop.io.compress.Lz4Codec
11/12/15 13:04:27 INFO snappy compress origin/compressed time: 0.13551s size:
100.000M/18.952M speed: 737.96M/139.86M
11/12/15 13:04:27 INFO snappy decompress orig/uncompressed time: 0.03126s size:
18.952M/100.000M speed: 606.24M/3198.87M
11/12/15 13:04:27 INFO ratio: 0.190
[ OK ] Perf.Compressions (4665 ms)
[----------] 1 test from Perf (4665 ms total)
> Add support for LZ4 compression
> -------------------------------
>
> Key: HADOOP-7657
> URL: https://issues.apache.org/jira/browse/HADOOP-7657
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Mr Bsd
> Labels: compression
>
> According to several benchmark sites, LZ4 seems to overtake other fast
> compression algorithms, especially in the decompression speed area. The
> interface is also trivial to integrate
> (http://code.google.com/p/lz4/source/browse/trunk/lz4.h) and there is no
> license issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
