[
https://issues.apache.org/jira/browse/MAPREDUCE-1743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13269066#comment-13269066
]
Jim Donofrio commented on MAPREDUCE-1743:
-----------------------------------------
private void updateJobWithSplit(final JobConf job, InputSplit inputSplit) {
if (inputSplit instanceof FileSplit) {
FileSplit fileSplit = (FileSplit) inputSplit;
job.set(JobContext.MAP_INPUT_FILE, fileSplit.getPath().toString());
job.setLong(JobContext.MAP_INPUT_START, fileSplit.getStart());
job.setLong(JobContext.MAP_INPUT_PATH, fileSplit.getLength());
} else if (inputSplit instanceof TaggedInputSplit) {
updateJobWithSplit(job, ((TaggedInputSplit) inputSplit).getInputSplit());
}
}
If make TaggedInputSplit a public class instead of package private then you can
change updateJobWithSplit to simply call itself again if the inputSplit is a
TaggedInputSplit.
> conf.get("map.input.file") returns null when using MultipleInputs in Hadoop
> 0.20
> --------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1743
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1743
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.2
> Reporter: Yuanyuan Tian
>
> There is a problem in getting the input file name in the mapper when uisng
> MultipleInputs in Hadoop 0.20. I need to use MultipleInputs to support
> different formats for my inputs to the my MapReduce job. And inside each
> mapper, I also need to know the exact input file that the mapper is
> processing. However, conf.get("map.input.file") returns null. Can anybody
> help me solve this problem? Thanks in advance.
> public class Test extends Configured implements Tool{
> static class InnerMapper extends MapReduceBase implements
> Mapper<Writable, Writable, NullWritable, Text>
> {
> ................
> ................
> public void configure(JobConf conf)
> {
> String inputName=conf.get("map.input.file"));
> .......................................
> }
>
> }
>
> public int run(String[] arg0) throws Exception {
> JonConf job;
> job = new JobConf(Test.class);
> ...........................................
>
> MultipleInputs.addInputPath(conf, new Path("A"),
> TextInputFormat.class);
> MultipleInputs.addInputPath(conf, new Path("B"),
> SequenceFileFormat.class);
> ...........................................
> }
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
