[
https://issues.apache.org/jira/browse/MESOS-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14969382#comment-14969382
]
Klaus Ma commented on MESOS-3765:
---------------------------------
[~alexr], I think we can use {{requestResources()}} to send granularity; so
allocator send offer based on granularity in each framework (the granularity
maybe different within one framework). Here's a workflow in allocator:
1. if there's no available resources to allocate, return
2. for each framework, allocate "granularity" resources in each loop, the
"granularity" will be a.) "total/framework #" for each resource type
(cput/mem), if no "granularity" from framework; b.) the "granularity" from
framework sent by {{requestResources()}}
3. keep running #2 until no available resources.
The frame work will fair-share the resource in the cluster, although greedy
framework may send several "granularity" to allocator; because allocator will
give only one "granularity" to framework in each loop. There is still a case
that framework overcommit by "granularity" (granularity > required resources )
which we can not guarantee, because allocator did not know framework's task
size.
*Demonstration:*
Two framework(f1 and f2), each of them will acquire 1 CPU; one slave with only
1 CPU; so the workflow will be:
- each framework will receive 0.5 CPU because no "granularity" from
framework, the default "granularity" is 1 CPU /2 framework.
- each framework will send "granularity" by {{requestResources}} for 1 CPU,
because the offer can not meet the requirement (1 vs. 0.5)
- allocator will send the 1 CPU to f1 (or f2) based on its "granularity"
One concern is performance downgrade, we need a perf test for it; and maybe a
flag to control this behaviour.
If any comments, please let me know.
> Make offer size adjustable (granularity)
> ----------------------------------------
>
> Key: MESOS-3765
> URL: https://issues.apache.org/jira/browse/MESOS-3765
> Project: Mesos
> Issue Type: Improvement
> Components: allocation
> Reporter: Alexander Rukletsov
>
> The built-in allocator performs "coarse-grained" allocation, meaning that it
> always allocates the entire remaining agent resources to a single framework.
> This may heavily impact allocation fairness in some cases, for example in
> presence of numerous greedy frameworks and a small number of powerful agents.
> A possible solution would be to allow operators explicitly specify
> granularity via allocator flags. While this can be tricky for non-standard
> resources, it's pretty straightforward for {{cpus}} and {{mem}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
