Hi,
The allocatable
<https://github.com/apache/mesos/blob/1.5.x/src/master/allocator/mesos/hierarchical.cpp#L2471-L2479>
check in the allocator (shown below) was originally introduced to help
alleviate the situation where a framework receives some resources, but no
cpu/memory, thus cannot launch a task.
bool HierarchicalAllocatorProcess::allocatable(
const Resources& resources)
{
Option<double> cpus = resources.cpus();
Option<Bytes> mem = resources.mem();
return (cpus.isSome() && cpus.get() >= MIN_CPUS) ||
(mem.isSome() && mem.get() >= MIN_MEM);
}
As pointed by Benjamin in MESOS-7398
<https://issues.apache.org/jira/browse/MESOS-7398>, it now seems to mainly
help to minimize the performance overhead from too many small offers
(instead too small resource amounts are kept out of the offer pool until
they became accumulated into larger resources).
This check does cause issues when new resources types are introduced. For
instance, this check does prevent GPU resources alone from being allocated
to a framework. There are some other issues we discover MESOS-8626
<https://issues.apache.org/jira/browse/MESOS-8626>.
There are several proposals:
(1) *Completely remove this check*. This check is a heuristic anyway, and
only applies to a subset of resources (cpu/memory). However, there might be
some implication of that change since it's also leveraged to prevent too
many small offers. *If you are concerned about this approach, please raise
your voice.*
(2) *Consider adjust the check to the following. *
bool HierarchicalAllocatorProcess::allocatable(
const Resources& resources)
{
Option<double> cpus = resources.cpus();
Option<Bytes> mem = resources.mem();
if (cpus.isSome() && mem.isSome()) {
return cpus.get() >= MIN_CPUS || mem.get() >= MIN_MEM;
} else if (cpus.isSome()) {
return cpus.get() >= MIN_CPUS;
} else if (mem.isSome()) {
return mem.get() >= MIN_MEM;
} else {
return true;
}
}
Let me know what you think! Thanks!
- Jie
