[
https://issues.apache.org/jira/browse/HADOOP-2639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12563294#action_12563294
]
Arun C Murthy commented on HADOOP-2639:
---------------------------------------
To be honest the details are hazy, but I do remember us considering writing a
simple loop to figure out that count rather than use _runningReduceTasks_;
anyway it does seem right that the denominator should be 'task-attempts' rather
than TIPs since the numerator is task-attempts too... Owen pointed this too.
The original patch needs to incorporate Devaraj's comment and I also need to
check if running{Map|Reduce}Tasks variables need to fiddled with in
JobInProgress.completedTask. There is a if-check where we check if the TIP is
already complete there, if so we don't manipulate the aforementioned
variables... I'll double check.
> Reducers stuck in shuffle
> -------------------------
>
> Key: HADOOP-2639
> URL: https://issues.apache.org/jira/browse/HADOOP-2639
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Amareshwari Sri Ramadasu
> Assignee: Amar Kamat
> Priority: Blocker
> Fix For: 0.16.0
>
> Attachments: HADOOP-2639.patch
>
>
> I started sort benchmark on 500 nodes. It has 40000 maps and 900 reducers.
> There are 11 reducers stuck in shuffle with 33% progress. I could see a node
> down which ran 80 maps on it. And all these reducers are trying to fetch map
> output from that node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
