Change fair scheduler to share between pools by default, not between invidual
jobs
----------------------------------------------------------------------------------
Key: HADOOP-4789
URL: https://issues.apache.org/jira/browse/HADOOP-4789
Project: Hadoop Core
Issue Type: Improvement
Components: contrib/fair-share
Reporter: Matei Zaharia
Priority: Minor
The fair scheduler currently treats jobs as equal entities in sharing by
default, so that a user who submits 2 jobs gets 2x the share of a user who
submits only 1 job. We found that it makes more sense to support equal shares
between individual pools instead, and have one pool per user, because users can
otherwise game the system by submitting multiple small jobs. This patch will
set the scheduler to share between pools by default and set the default pool
assignment process to one pool per user. it will also be possible to give
weights to pools so that some users/groups/applications get a larger share of
the cluster if they really do need to run more jobs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
