[
https://issues.apache.org/jira/browse/MAPREDUCE-6155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allen Wittenauer updated MAPREDUCE-6155:
----------------------------------------
Fix Version/s: (was: trunk)
> MapFiles are not always correctly detected by SequenceFileInputFormat
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-6155
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6155
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: trunk
> Reporter: Jens Rabe
> Attachments: MAPREDUCE-6155.001.patch, MAPREDUCE-6155.002.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> MapFiles are not correctly detected by SequenceFileInputFormat.
> This is because the listStatus method only detects a MapFile correctly if the
> path it checks is a directory - it then replaces it by the path of the data
> file.
> This is likely to fail if the data file does not exist, i.e., if the input
> path is a directory, but does not belong to a MapFile, or if recursion is
> turned on and the input format comes across a file (not a directory) which is
> indeed part of a MapFile.
> The listStatus method should be changed to detect these cases correctly:
> * if the current candidate is a file and its name is "index" or "data", check
> if its corresponding other file exists, and if the key types of both files
> match and if the value type of the index file is LongWritable
> * If the current candidate is a directory, it is only a MapFile if (and only
> if) an index and a data file exist, they are both SequenceFiles and their key
> types match (and the index value type is LongWritable)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
