[
https://issues.apache.org/jira/browse/HADOOP-1152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12490292
]
Devaraj Das commented on HADOOP-1152:
-------------------------------------
+1, regardless of the race condition theory, it makes sense to not call
getLength on a renamed ramfs file (that the merge thread might have already
deleted in the time interval between rename and getLength). But it seems very
likely the race condition is caused due to this.
> Reduce task hang failing in MapOutputCopier.copyOutput
> ------------------------------------------------------
>
> Key: HADOOP-1152
> URL: https://issues.apache.org/jira/browse/HADOOP-1152
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Reporter: Koji Noguchi
> Assigned To: Tahir Hashmi
> Attachments: 1152.patch, 1152.workaround.patch
>
>
> We had couple of reduce tasks hang repeating the output below.
> 2007-03-22 23:57:16,296 WARN org.apache.hadoop.mapred.TaskRunner:
> java.io.IOException: Path
> /hadoop/mapred/local/task_0026_r_000307_0/map_7854.out already exists
> at
> org.apache.hadoop.fs.InMemoryFileSystem$RawInMemoryFileSystem.rename(InMemoryFileSystem.java:246)
> at
> org.apache.hadoop.fs.ChecksumFileSystem.rename(ChecksumFileSystem.java:471)
> at
> org.apache.hadoop.mapred.ReduceTaskRunner$MapOutputCopier.copyOutput(ReduceTaskRunner.java:336)
> at
> org.apache.hadoop.mapred.ReduceTaskRunner$MapOutputCopier.run(ReduceTaskRunner.java:274)
> 2007-03-22 23:57:16,296 WARN org.apache.hadoop.mapred.TaskRunner:
> task_0026_r_000307_0 adding host ______ to penalty box, next contact in 192
> seconds
> ===============================
> Before the above output, there was
> 2007-03-22 18:15:24,274 ERROR org.apache.hadoop.mapred.TaskRunner: Map output
> copy failure: java.lang.NullPointerException
> at
> org.apache.hadoop.fs.InMemoryFileSystem$RawInMemoryFileSystem$FileAttributes.access$300(InMemoryFileSystem.java:416)
> at
> org.apache.hadoop.fs.InMemoryFileSystem$RawInMemoryFileSystem.getLength(InMemoryFileSystem.java:286)
> at
> org.apache.hadoop.fs.FilterFileSystem.getLength(FilterFileSystem.java:178)
> at
> org.apache.hadoop.mapred.ReduceTaskRunner$MapOutputCopier.copyOutput(ReduceTaskRunner.java:340)
> at
> org.apache.hadoop.mapred.ReduceTaskRunner$MapOutputCopier.run(ReduceTaskRunner.java:274)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
