[
https://issues.apache.org/jira/browse/HADOOP-4057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12627899#action_12627899
]
Owen O'Malley commented on HADOOP-4057:
---------------------------------------
I'd suggest something like this:
{code}
public class MultipleFileInputFormat implements InputFormat {
public static void setUserInputFormat(Configuration conf, Class<? extends
InputFormat> inputFormat);
public static Class<? extends InputFormat> getUserInputFormat(Configuration
conf);
public InputSplit[] getSplits(JobConf conf, int maps) {
InputFormat userInputFormat =
ReflectionUtils.newInstance(getUserInputFormat(conf), conf);
InputSplit[] userSplits = userInputFormat.getSPlits(conf, maps);
... merge them together ...
return exactly maps splits
}
public RecordReader getRecordReader(InputSplit split, JobConf conf, Reporter
reporter) {
.. return a record reader that returns all of the records for split 1, split
2, etc..
}
}
{code}
> MultipleFileInputFormat should support a sub InputFormat
> --------------------------------------------------------
>
> Key: HADOOP-4057
> URL: https://issues.apache.org/jira/browse/HADOOP-4057
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Owen O'Malley
>
> Currently, to use MultipleFileInputFormat you need to create a subclass, but
> it seems pretty easy to modify it to use a sub InputFormat and put the
> results together into bigger splits.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
