[
https://issues.apache.org/jira/browse/HADOOP-4640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Johan Oskarsson updated HADOOP-4640:
------------------------------------
Attachment: (was: HADOOP-4622.patch)
> Add ability to split text files compressed with lzo
> ---------------------------------------------------
>
> Key: HADOOP-4640
> URL: https://issues.apache.org/jira/browse/HADOOP-4640
> Project: Hadoop Core
> Issue Type: Improvement
> Components: io, mapred
> Reporter: Johan Oskarsson
> Assignee: Johan Oskarsson
> Priority: Trivial
> Fix For: 0.20.0
>
> Attachments: HADOOP-4640.patch
>
>
> Right now any file compressed with lzop will be processed by one mapper. This
> is a shame since the lzo algorithm would be very suitable for large log files
> and similar common hadoop data sets. The compression rate is not the best out
> there but the decompression speed is amazing. Since lzo writes compressed
> data in blocks it would be possible to make an input format that can split
> the files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
