[ http://issues.apache.org/jira/browse/HADOOP-343?page=all ]
Mahadev konar updated HADOOP-343:
---------------------------------
Attachment: bugfix.patch
This patch updates its mapoutput locations in case of any failures on copying
from the task tracker. So, in case a copy failed from a task tracker, the map
outputs corresponding to this node will be marked stale. The ReduceTask will
ask for the mapout locations again for these stale map outputs. This patch also
fixes a bug wherein the tasks keep polling the job tracker for map otputs "in a
loop" without sleeping/waiting. With this fix the tasks will poll evry
MIN_POLL_INTERVAL before querying the job tracker.
> In case of dead task tracker, the copy mapouts try copying all mapoutputs
> from this tasktracker
> -----------------------------------------------------------------------------------------------
>
> Key: HADOOP-343
> URL: http://issues.apache.org/jira/browse/HADOOP-343
> Project: Hadoop
> Type: Bug
> Components: mapred
> Versions: 0.5.0
> Reporter: Mahadev konar
> Assignee: Mahadev konar
> Attachments: bugfix.patch
>
> In case of a dead task tracker, the reduces which do not have the updated map
> out locations try copygin files from this node and since there are failures
> on copying, this leads to backoff and slowing down of the copy pahse.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
