[
https://issues.apache.org/jira/browse/HADOOP-4789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12662160#action_12662160
]
Matei Zaharia commented on HADOOP-4789:
---------------------------------------
I just committed this. The two failed tests were a deadlock in unrelated code
due to https://issues.apache.org/jira/browse/HADOOP-4977 and a chukwa test
which now passes. We have also been running this in production at Facebook.
> Change fair scheduler to share between pools by default, not between invidual
> jobs
> ----------------------------------------------------------------------------------
>
> Key: HADOOP-4789
> URL: https://issues.apache.org/jira/browse/HADOOP-4789
> Project: Hadoop Core
> Issue Type: Improvement
> Components: contrib/fair-share
> Reporter: Matei Zaharia
> Assignee: Matei Zaharia
> Priority: Minor
> Fix For: 0.19.1
>
> Attachments: hadoop-4789-v1.patch
>
>
> The fair scheduler currently treats jobs as equal entities in sharing by
> default, so that a user who submits 2 jobs gets 2x the share of a user who
> submits only 1 job. We found that it makes more sense to support equal shares
> between individual pools instead, and have one pool per user, because users
> can otherwise game the system by submitting multiple small jobs. This patch
> will set the scheduler to share between pools by default and set the default
> pool assignment process to one pool per user. it will also be possible to
> give weights to pools so that some users/groups/applications get a larger
> share of the cluster if they really do need to run more jobs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
