[
https://issues.apache.org/jira/browse/HADOOP-2639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12563374#action_12563374
]
Amar Kamat commented on HADOOP-2639:
------------------------------------
Wondering about the last comment in the patch. After going over the JT and JIP
code multiple times we found that the code
{code}
// Sanity check: is the TIP already complete?
if (tip.isComplete()) {
// Mark this task as KILLED
tip.alreadyCompletedTask(taskid);
// Let the JobTracker cleanup this taskid if the job isn't running
if (this.status.getRunState() != JobStatus.RUNNING) {
jobtracker.markCompletedTaskAttempt(status.getTaskTracker(), taskid);
}
return false;
}
{code}
will never be executed since {{completedTask()}} is called once for a TIP and
that too for the first _SUCCEEDED (COMMIT_PENDING -> SUCCEEDED)_ task and
hence {{tip.isComplete()}} will always be false in this case, no?
> Reducers stuck in shuffle
> -------------------------
>
> Key: HADOOP-2639
> URL: https://issues.apache.org/jira/browse/HADOOP-2639
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amareshwari Sri Ramadasu
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.16.0
>
> Attachments: HADOOP-2639.patch, HADOOP-2639_1_20080128.patch
>
>
> I started sort benchmark on 500 nodes. It has 40000 maps and 900 reducers.
> There are 11 reducers stuck in shuffle with 33% progress. I could see a node
> down which ran 80 maps on it. And all these reducers are trying to fetch map
> output from that node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
