Todd, you had some questions about splittability in the Jira.
To what extent do Sequence Files remove the need for codec-level splittability?
I'm working on gzip splittability in hadoop-7909, which could be applied to any
compression method we want to assign a CM value for in the gzip header, but I'm
unclear on how much this general-purpose splittability is redundant to sequence
files.
- Tim.
On Dec 14, 2011, at 10:52 PM, "Todd Lipcon (Commented) (JIRA)"
<[email protected]> wrote:
>
> [
> https://issues.apache.org/jira/browse/HADOOP-7657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169994#comment-13169994
> ]
>
> Todd Lipcon commented on HADOOP-7657:
> -------------------------------------
>
> great results - looks like about 15% fsater compression and 60% faster
> decompression than snappy, along with a bit better ratio?
>
> Do you have the codec up in a patch somewhere for others to try? Also, I'm
> assuming you ran the test in a loop several times in the same JVM to let the
> JIT warm up equally for both algorithms?
>
>> Add support for LZ4 compression
>> -------------------------------
>>
>> Key: HADOOP-7657
>> URL: https://issues.apache.org/jira/browse/HADOOP-7657
>> Project: Hadoop Common
>> Issue Type: Improvement
>> Reporter: Mr Bsd
>> Labels: compression
>>
>> According to several benchmark sites, LZ4 seems to overtake other fast
>> compression algorithms, especially in the decompression speed area. The
>> interface is also trivial to integrate
>> (http://code.google.com/p/lz4/source/browse/trunk/lz4.h) and there is no
>> license issue.
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA
> administrators:
> https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
>
The information and any attached documents contained in this message
may be confidential and/or legally privileged. The message is
intended solely for the addressee(s). If you are not the intended
recipient, you are hereby notified that any use, dissemination, or
reproduction is strictly prohibited and may be unlawful. If you are
not the intended recipient, please contact the sender immediately by
return e-mail and destroy all copies of the original message.
