Chatted with BenM offline on this. There's another option what both of us agreed that it's probably better than any of the ones mentioned above.
The idea is to make `allocable` return the portion of the input resources that are allocatable, and strip the unelectable portion. For example: 1) If the input resources are "cpus:0.001,gpus:1", the `allocatable` method will return "gpus:1". 2) If the input resources are "cpus:1,mem:1", the `allocatable` method will return "cpus:1". 3) If the input resources are "cpus:0.001,mem:1", the `allocatable` method will return an empty Resources object. Basically, the algorithm is like the following: allocatable = input foreach known resource type t: do r = resources of type t from the input if r is less than the min resource of type t; then allocatable -= r fi done return allocatable Let me know what do you guys think! Thanks! - Jie On Fri, Mar 2, 2018 at 4:44 PM, Benjamin Mahler <bmah...@apache.org> wrote: > I think (2) would need to be: > > bool HierarchicalAllocatorProcess::allocatable( > const Resources& resources) > { > if (something outside {cpu, mem, disk} is present) return true > else return true iff at least one of {cpu, mem, disk} has >= {MIN_CPU, > MIN_MEM, MIN_DISK} > } > > Otherwise, 1 GPU would be offered but 1GPU + 0.001 CPU would not? > > On Fri, Mar 2, 2018 at 9:27 AM, Jie Yu <yujie....@gmail.com> wrote: > > > Hi, > > > > The allocatable > > <https://github.com/apache/mesos/blob/1.5.x/src/master/alloc > > ator/mesos/hierarchical.cpp#L2471-L2479> > > check in the allocator (shown below) was originally introduced to help > > alleviate the situation where a framework receives some resources, but no > > cpu/memory, thus cannot launch a task. > > > > bool HierarchicalAllocatorProcess::allocatable( > > const Resources& resources) > > { > > Option<double> cpus = resources.cpus(); > > Option<Bytes> mem = resources.mem(); > > > > return (cpus.isSome() && cpus.get() >= MIN_CPUS) || > > (mem.isSome() && mem.get() >= MIN_MEM); > > } > > > > As pointed by Benjamin in MESOS-7398 > > <https://issues.apache.org/jira/browse/MESOS-7398>, it now seems to > mainly > > help to minimize the performance overhead from too many small offers > > (instead too small resource amounts are kept out of the offer pool until > > they became accumulated into larger resources). > > > > This check does cause issues when new resources types are introduced. For > > instance, this check does prevent GPU resources alone from being > allocated > > to a framework. There are some other issues we discover MESOS-8626 > > <https://issues.apache.org/jira/browse/MESOS-8626>. > > > > There are several proposals: > > > > (1) *Completely remove this check*. This check is a heuristic anyway, and > > only applies to a subset of resources (cpu/memory). However, there might > be > > some implication of that change since it's also leveraged to prevent too > > many small offers. *If you are concerned about this approach, please > raise > > your voice.* > > > > (2) *Consider adjust the check to the following. * > > > > bool HierarchicalAllocatorProcess::allocatable( > > const Resources& resources) > > { > > Option<double> cpus = resources.cpus(); > > Option<Bytes> mem = resources.mem(); > > > > if (cpus.isSome() && mem.isSome()) { > > return cpus.get() >= MIN_CPUS || mem.get() >= MIN_MEM; > > } else if (cpus.isSome()) { > > return cpus.get() >= MIN_CPUS; > > } else if (mem.isSome()) { > > return mem.get() >= MIN_MEM; > > } else { > > return true; > > } > > } > > > > Let me know what you think! Thanks! > > > > - Jie > > >