[
https://issues.apache.org/jira/browse/HADOOP-1862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun C Murthy updated HADOOP-1862:
----------------------------------
Assignee: Arun C Murthy (was: Owen O'Malley)
Ok, here is the proposal to fix this one:
a) Maintain a straight cache at the {{TaskTracker}} i.e. straight copy of the
events sent by the {{JobTracker}} and don't manipulate them.
b) Process {{OBSOLETE}} {{TaskCompletionEvent}}s at the {{ReduceTask}}.
c) Keep my earlier fix to {{TaskInProgress.completedTask}} i.e. do not add a
{{SUCCESS}} event until it is clear that this is the *true* successful event,
and that the TIP hadn't already been completed by some other task-attempt.
> reduces are getting stuck trying to find map outputs
> ----------------------------------------------------
>
> Key: HADOOP-1862
> URL: https://issues.apache.org/jira/browse/HADOOP-1862
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.13.0, 0.13.1, 0.14.0, 0.14.1
> Reporter: Owen O'Malley
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 14.2
>
> Attachments: HADOOP-1862_debug.patch, HADOOP-1862_prelim.patch
>
>
> Some of the reduces have been stuck for hours looking for 137 map outputs.
> When I look at the job events all 2600 of the maps have succeeded. There have
> been lots of lost task trackers and shuffle failures. The maps have been run
> between 1 to 6 times each. I do see some of the events in the task event log
> are marked OBSOLETE.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
