[
https://issues.apache.org/jira/browse/HADOOP-5160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12670315#action_12670315
]
Vinod K V commented on HADOOP-5160:
-----------------------------------
Nathan, what scheduler are you using? Assuming it is the default one, the
default scheduler does have load balancing code to distribute map *and* reduce
tasks evenly on all the TaskTtrackers depending on each TT's slots. Can you
give more information about your observation - things like hadoop version, size
of your cluster, more info about your job etc.?
> Hadoop reduce scheduler sometimes leaves machines idle
> ------------------------------------------------------
>
> Key: HADOOP-5160
> URL: https://issues.apache.org/jira/browse/HADOOP-5160
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Nathan Marz
>
> I have a MapReduce application with number of reducers equal to the number of
> machines in the cluster (and with speculative execution turned off). However,
> Hadoop schedules multiple reduces to run on single machines and leaves other
> machines idle. This causes contention and seriously slows down the job.
> Hadoop should employ the simple heuristic of utilizing as many machines as
> possible when scheduling reduces.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
