[
https://issues.apache.org/jira/browse/HADOOP-3236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12588089#action_12588089
]
Devaraj Das commented on HADOOP-3236:
-------------------------------------
Actually, whenever there is an error in the merge, the InMemoryFileSystem is
left in a state where other threads using the InMemoryFileSystem would
encounter NPEs. This needs to be fixed. But it's not a correctness issue since
if there is a merge exception the task will die for sure.
So yes the NPEs are due to the other merge exceptions as you pointed out. But I
am not sure why so many reducers are seeing corrupted map outputs.
> Reduce task failed at shuffling time, throwing null pointer exception
> ---------------------------------------------------------------------
>
> Key: HADOOP-3236
> URL: https://issues.apache.org/jira/browse/HADOOP-3236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Runping Qi
>
> This happened for 0.17.0 branch.
> Here is the stack trace:
> 2008-04-11 13:45:54,171 ERROR org.apache.hadoop.mapred.ReduceTask: Map output
> copy failure: java.lang.NullPointerException
> at
> org.apache.hadoop.fs.InMemoryFileSystem$RawInMemoryFileSystem.getFileStatus(InMemoryFileSystem.java:302)
> at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:242)
> at
> org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.copyOutput(ReduceTask.java:853)
> at
> org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.run(ReduceTask.java:777)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
