[
https://issues.apache.org/jira/browse/HADOOP-5160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12669865#action_12669865
]
Bryan Duxbury commented on HADOOP-5160:
---------------------------------------
It seems like it should be easy enough to pass out reduce tasks to a machine
one at a time. Even with speculative execution on, this should improve the
distribution of tasks.
> Hadoop reduce scheduler sometimes leaves machines idle
> ------------------------------------------------------
>
> Key: HADOOP-5160
> URL: https://issues.apache.org/jira/browse/HADOOP-5160
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Nathan Marz
>
> I have a MapReduce application with number of reducers equal to the number of
> machines in the cluster (and with speculative execution turned off). However,
> Hadoop schedules multiple reduces to run on single machines and leaves other
> machines idle. This causes contention and seriously slows down the job.
> Hadoop should employ the simple heuristic of utilizing as many machines as
> possible when scheduling reduces.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
