[
https://issues.apache.org/jira/browse/MAPREDUCE-1597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amareshwari Sriramadasu updated MAPREDUCE-1597:
-----------------------------------------------
Attachment: patch-1597.txt
Patch adding the support for non-splittable files in CombineFileInputFormat. If
the file is not splittable, it generates OneBlockInfo with full file length.
> combinefileinputformat does not work with non-splittable files
> --------------------------------------------------------------
>
> Key: MAPREDUCE-1597
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1597
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Namit Jain
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.22.0
>
> Attachments: patch-1597.txt
>
>
> CombineFileInputFormat.getSplits() does not take into account whether a file
> is splittable.
> This can lead to a problem for compressed text files - for example,
> getSplits() may return more
> than 1 split depending on the size of the compressed file, all the splits
> recordreader will read the
> complete file.
> I ran into this problem while using Hive on hadoop 20.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
