[
https://issues.apache.org/jira/browse/AURORA-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620950#comment-14620950
]
Anindya Sinha commented on AURORA-1384:
---------------------------------------
Thanks Bill for your comments. Couple of points to consider though before
deciding that this may or may not be needed.
Considering a generic solution, I think clusters can have variable degree of
traffic such as high rate of jobs, different resource requirements for jobs,
etc. In cases where cluster do not have minimal churn, we can land into
scenarios where specific slaves are using bulk of their "total offer" whereas
there are other slaves in the cluster which are relatively "free".
In my specific scenario, we have quite a bit of churn in our cluster which
results in variable job acceptance rate as well as variable size of jobs. If
the "highly loaded" slaves end up with all production jobs, that would result
in no space for more jobs in that slave which is a problem for say "ubiquitous
jobs" (as an example, AURORA-1075).
I would still encourage to consider this jira. Keeping the existing policy
(which I refer as "first-fit") can be a default with additional policies such
as "bin-packed" and "even-distribution". Once we have this, we can add more
constraints to this such as to avoid a large %age of production jobs on any
single node, and so on.
> Provide alternate policies for offer assignment to tasks
> --------------------------------------------------------
>
> Key: AURORA-1384
> URL: https://issues.apache.org/jira/browse/AURORA-1384
> Project: Aurora
> Issue Type: Story
> Components: Scheduler
> Affects Versions: 0.8.0
> Reporter: Anindya Sinha
>
> Currently, offers are arranged in a list. When new tasks need to be assigned
> an offer, the first offer from the list that matches the task constraints is
> assigned which over time may result in a non even distribution of tasks on
> slaves in the cluster.
> In addition to existing algorithm ("first-fit"), we can support alternate
> algorithms as follows:
> i) bin-packed: Offer is assigned to a task based on a bin packed algorithm
> which ensures a specific offer is used up to the maximum possible with
> respect to resources (cpu) before moving to an alternate offer.
> ii) even-distribution: Offer is assigned to ensure even distribution of load
> on each slave on the cluster based on resources (cpu).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
