Re: Reconsidering `allocatable` check in the allocator

[Zhitao Li]

To register our possible use case which also requires relaxing this check:

We are looking at whether it's feasible to use a custom resource type to
present additional tasks in a group. This means our scheduler could add a
TaskGroup with no cpu/mem to an existing executor. Because 1) new TaskGroup
cannot have empty resource and 2) it could have new routing primitive, we
might add several port(s) as only resources of the group.

To cover all corner cases, we'd like to ensure that framework would always
see an offer from an agent, even if it has no cpu/mem left to allocate but
only some ports. (In our setup, number of ports per host is much bigger
than possible density of tasks so we never run out of ports at any host).

I think I can see people who uses custom resource types could face have
similar issue caused by this `allocatable` check.

Thanks.

On Wed, Mar 7, 2018 at 5:29 PM, Benjamin Mahler  wrote:

> +1 about it not being about network traffic.
>
> I the direction we want to head towards is to express and enforce a minimum
> granularity for scalar resources. For example:
>
> CPU: 0.001, if we say that we can only deal with milli-cpus.
> Disk: 1, if we say that we can only deal with the MB level of disk space
> isolation.
> GPU: 1, we can't let you consume a portion of a GPU.
>
> Note that this issue is caused by the lack of an "Integer" resource,
> because with an "Integer" resource we can just store the value based on the
> minimum granularity (e.g. 1 milli-cpus, 1 byte disk, etc). Note also that
> with Scalar resources, we currently only support three decimal points of
> precision:
> https://github.com/apache/mesos/blob/1.5.0/include/
> mesos/v1/mesos.proto#L1098-L1108
>
> If you were to check the minimum granularity on the input side (e.g.
> prevent frameworks from taking 4.0001 of 8 cpus in an offer), then you
> don't technically need to prevent the allocator from violating the minimum
> granularity (e.g. because we prevent 1.1 or 0.5 GPUs on the input side,
> only whole numbers of GPUs will be available).
>
> Re: Filtering discussion
>
> I think this is a separate discussion, worth having, about how to let
> frameworks give more information about what they want in order to tame the
> churn of offers.
>
> On Wed, Mar 7, 2018 at 9:53 AM, James Peach  wrote:
>
> >
> >
> > > On Mar 7, 2018, at 5:52 AM, Benjamin Bannier <
> > benjamin.bann...@mesosphere.io> wrote:
> > >
> > > Hi,
> > >
> > >> Chatted with BenM offline on this. There's another option what both of
> > us
> > >> agreed that it's probably better than any of the ones mentioned above.
> > >>
> > >> The idea is to make `allocable` return the portion of the input
> > resources
> > >> that are allocatable, and strip the unelectable portion.
> > >>
> > >> For example:
> > >> 1) If the input resources are "cpus:0.001,gpus:1", the `allocatable`
> > method
> > >> will return "gpus:1".
> > >> 2) If the input resources are "cpus:1,mem:1", the `allocatable` method
> > will
> > >> return "cpus:1".
> > >> 3) If the input resources are "cpus:0.001,mem:1", the `allocatable`
> > method
> > >> will return an empty Resources object.
> > >>
> > >> Basically, the algorithm is like the following:
> > >>
> > >> allocatable = input
> > >> foreach known resource type t: do
> > >> r = resources of type t from the input
> > >> if r is less than the min resource of type t; then
> > >>   allocatable -= r
> > >> fi
> > >> done
> > >> return allocatable
> > >
> > > I think that sounds like a faithful extension the current behavior to
> me
> > (removing too small resources from the offerable pool), but I feel we
> > should not just filter out any resource _kind_  below the minimum, but
> > inside a kind all _addable_ subresources,
> > >
> > >allocatable : Resources = input
> > >  for (resource: Resource) in input:
> > >if resource < min(resource.kind):
> > >  allocatable -= resource
> > >
> > >return allocatable
> > >
> > > This would have the effect of clumping together each distinguishable
> > resource we care about instead of of accumulating say different disks
> which
> > in sum are potentially not that more interesting to frameworks (they
> would
> > prefer more of a particular disk than smaller pieces scattered across
> > multiple disks).
> > >
> > > @alexr
> > >> If we are about to offer some of the resources from a particular
> agent,
> > why
> > >> would we filter anything at all? I doubt we should be concerned about
> > the
> > >> size of the offer representation travelling through the network. If
> > >> available resources are "cpus:0.001,gpus:1" and we want to allocate
> GPU,
> > >> what is the benefit of filtering CPU?
> > >>
> > >> What about the following:
> > >> allocatable(R)
> > >> {
> > >> return true
> > >>   iff (there exists r in R for which size(r) > MIN(type(r)))
> > >> }
> > >
> > > I think this is less about communication overhead, but more a tool to
> > help to make sure that 

Re: Reconsidering `allocatable` check in the allocator

[Benjamin Mahler]

+1 about it not being about network traffic.

I the direction we want to head towards is to express and enforce a minimum
granularity for scalar resources. For example:

CPU: 0.001, if we say that we can only deal with milli-cpus.
Disk: 1, if we say that we can only deal with the MB level of disk space
isolation.
GPU: 1, we can't let you consume a portion of a GPU.

Note that this issue is caused by the lack of an "Integer" resource,
because with an "Integer" resource we can just store the value based on the
minimum granularity (e.g. 1 milli-cpus, 1 byte disk, etc). Note also that
with Scalar resources, we currently only support three decimal points of
precision:
https://github.com/apache/mesos/blob/1.5.0/include/mesos/v1/mesos.proto#L1098-L1108

If you were to check the minimum granularity on the input side (e.g.
prevent frameworks from taking 4.0001 of 8 cpus in an offer), then you
don't technically need to prevent the allocator from violating the minimum
granularity (e.g. because we prevent 1.1 or 0.5 GPUs on the input side,
only whole numbers of GPUs will be available).

Re: Filtering discussion

I think this is a separate discussion, worth having, about how to let
frameworks give more information about what they want in order to tame the
churn of offers.

On Wed, Mar 7, 2018 at 9:53 AM, James Peach  wrote:

>
>
> > On Mar 7, 2018, at 5:52 AM, Benjamin Bannier <
> benjamin.bann...@mesosphere.io> wrote:
> >
> > Hi,
> >
> >> Chatted with BenM offline on this. There's another option what both of
> us
> >> agreed that it's probably better than any of the ones mentioned above.
> >>
> >> The idea is to make `allocable` return the portion of the input
> resources
> >> that are allocatable, and strip the unelectable portion.
> >>
> >> For example:
> >> 1) If the input resources are "cpus:0.001,gpus:1", the `allocatable`
> method
> >> will return "gpus:1".
> >> 2) If the input resources are "cpus:1,mem:1", the `allocatable` method
> will
> >> return "cpus:1".
> >> 3) If the input resources are "cpus:0.001,mem:1", the `allocatable`
> method
> >> will return an empty Resources object.
> >>
> >> Basically, the algorithm is like the following:
> >>
> >> allocatable = input
> >> foreach known resource type t: do
> >> r = resources of type t from the input
> >> if r is less than the min resource of type t; then
> >>   allocatable -= r
> >> fi
> >> done
> >> return allocatable
> >
> > I think that sounds like a faithful extension the current behavior to me
> (removing too small resources from the offerable pool), but I feel we
> should not just filter out any resource _kind_  below the minimum, but
> inside a kind all _addable_ subresources,
> >
> >allocatable : Resources = input
> >  for (resource: Resource) in input:
> >if resource < min(resource.kind):
> >  allocatable -= resource
> >
> >return allocatable
> >
> > This would have the effect of clumping together each distinguishable
> resource we care about instead of of accumulating say different disks which
> in sum are potentially not that more interesting to frameworks (they would
> prefer more of a particular disk than smaller pieces scattered across
> multiple disks).
> >
> > @alexr
> >> If we are about to offer some of the resources from a particular agent,
> why
> >> would we filter anything at all? I doubt we should be concerned about
> the
> >> size of the offer representation travelling through the network. If
> >> available resources are "cpus:0.001,gpus:1" and we want to allocate GPU,
> >> what is the benefit of filtering CPU?
> >>
> >> What about the following:
> >> allocatable(R)
> >> {
> >> return true
> >>   iff (there exists r in R for which size(r) > MIN(type(r)))
> >> }
> >
> > I think this is less about communication overhead, but more a tool to
> help to make sure that offered resources are actually useful to frameworks.
>
> I don't know whether there's a JIRA for this, but in the past we've
> proposed the idea of schedulers suppressing or filtering offers with a
> minimum resources specification, i.e. "don't bother me with offers that
> aren't at least X"
>
> J

Re: Reconsidering `allocatable` check in the allocator

[James Peach]

> On Mar 7, 2018, at 5:52 AM, Benjamin Bannier  
> wrote:
> 
> Hi,
> 
>> Chatted with BenM offline on this. There's another option what both of us
>> agreed that it's probably better than any of the ones mentioned above.
>> 
>> The idea is to make `allocable` return the portion of the input resources
>> that are allocatable, and strip the unelectable portion.
>> 
>> For example:
>> 1) If the input resources are "cpus:0.001,gpus:1", the `allocatable` method
>> will return "gpus:1".
>> 2) If the input resources are "cpus:1,mem:1", the `allocatable` method will
>> return "cpus:1".
>> 3) If the input resources are "cpus:0.001,mem:1", the `allocatable` method
>> will return an empty Resources object.
>> 
>> Basically, the algorithm is like the following:
>> 
>> allocatable = input
>> foreach known resource type t: do
>> r = resources of type t from the input
>> if r is less than the min resource of type t; then
>>   allocatable -= r
>> fi
>> done
>> return allocatable
> 
> I think that sounds like a faithful extension the current behavior to me 
> (removing too small resources from the offerable pool), but I feel we should 
> not just filter out any resource _kind_  below the minimum, but inside a kind 
> all _addable_ subresources,
> 
>allocatable : Resources = input
>  for (resource: Resource) in input:
>if resource < min(resource.kind):
>  allocatable -= resource
> 
>return allocatable
> 
> This would have the effect of clumping together each distinguishable resource 
> we care about instead of of accumulating say different disks which in sum are 
> potentially not that more interesting to frameworks (they would prefer more 
> of a particular disk than smaller pieces scattered across multiple disks).
> 
> @alexr
>> If we are about to offer some of the resources from a particular agent, why
>> would we filter anything at all? I doubt we should be concerned about the
>> size of the offer representation travelling through the network. If
>> available resources are "cpus:0.001,gpus:1" and we want to allocate GPU,
>> what is the benefit of filtering CPU?
>> 
>> What about the following:
>> allocatable(R)
>> {
>> return true
>>   iff (there exists r in R for which size(r) > MIN(type(r)))
>> }
> 
> I think this is less about communication overhead, but more a tool to help to 
> make sure that offered resources are actually useful to frameworks. 

I don't know whether there's a JIRA for this, but in the past we've proposed 
the idea of schedulers suppressing or filtering offers with a minimum resources 
specification, i.e. "don't bother me with offers that aren't at least X"

J

Re: Reconsidering `allocatable` check in the allocator

[Benjamin Bannier]

Hi,

> Chatted with BenM offline on this. There's another option what both of us
> agreed that it's probably better than any of the ones mentioned above.
> 
> The idea is to make `allocable` return the portion of the input resources
> that are allocatable, and strip the unelectable portion.
> 
> For example:
> 1) If the input resources are "cpus:0.001,gpus:1", the `allocatable` method
> will return "gpus:1".
> 2) If the input resources are "cpus:1,mem:1", the `allocatable` method will
> return "cpus:1".
> 3) If the input resources are "cpus:0.001,mem:1", the `allocatable` method
> will return an empty Resources object.
> 
> Basically, the algorithm is like the following:
> 
> allocatable = input
> foreach known resource type t: do
>  r = resources of type t from the input
>  if r is less than the min resource of type t; then
>allocatable -= r
>  fi
> done
> return allocatable

I think that sounds like a faithful extension the current behavior to me 
(removing too small resources from the offerable pool), but I feel we should 
not just filter out any resource _kind_  below the minimum, but inside a kind 
all _addable_ subresources,

allocatable : Resources = input
  for (resource: Resource) in input:
if resource < min(resource.kind):
  allocatable -= resource

return allocatable

This would have the effect of clumping together each distinguishable resource 
we care about instead of of accumulating say different disks which in sum are 
potentially not that more interesting to frameworks (they would prefer more of 
a particular disk than smaller pieces scattered across multiple disks).

@alexr
> If we are about to offer some of the resources from a particular agent, why
> would we filter anything at all? I doubt we should be concerned about the
> size of the offer representation travelling through the network. If
> available resources are "cpus:0.001,gpus:1" and we want to allocate GPU,
> what is the benefit of filtering CPU?
> 
> What about the following:
> allocatable(R)
> {
>  return true
>iff (there exists r in R for which size(r) > MIN(type(r)))
> }

I think this is less about communication overhead, but more a tool to help to 
make sure that offered resources are actually useful to frameworks. If we would 
completely remove the current behavior of clumping resources it might take a 
long time for frameworks to actually receive sufficient interesting resources. 
While frameworks can use filters to prevent some offers, to filter out an offer 
we currently always require that the filtered resources are a superset of the 
resources we are about to offer. As the number of possible dimensions (e.g., 
resource kinds, labels, other fields) increases it becomes harder and harder 
for filters to be effective in this regard and the allocator needs to step in.

https://en.wikipedia.org/wiki/Curse_of_dimensionality


Cheers,

Benjamin

Re: Reconsidering `allocatable` check in the allocator

[Alex Rukletsov]

If we are about to offer some of the resources from a particular agent, why
would we filter anything at all? I doubt we should be concerned about the
size of the offer representation travelling through the network. If
available resources are "cpus:0.001,gpus:1" and we want to allocate GPU,
what is the benefit of filtering CPU?

What about the following:
allocatable(R)
{
  return true
iff (there exists r in R for which size(r) > MIN(type(r)))
}

On Wed, Mar 7, 2018 at 2:41 AM, Qian Zhang  wrote:

> So if the input resources are "cpus:0.001,disk:1024", the `allocatable`
> method will return "disk:1024"? This seems not compatible with the existing
> behavior: with the current implementation of `allocatable`, the same input
> resources will be just skipped because we think "cpus:0.001" is too small
> for framework to launch a task.
>
> allocatable = input
> > foreach known resource type t: do
> >   r = resources of type t from the input
> >   if r is less than the min resource of type t; then
> > allocatable -= r
> >   fi
> > done
> > return allocatable
> >
>
> Are we going to define min amount for each known resource type (including
> disk and gpu)?
>
>
> Regards,
> Qian Zhang
>
> On Wed, Mar 7, 2018 at 6:10 AM, Jie Yu  wrote:
>
> > Chatted with BenM offline on this. There's another option what both of us
> > agreed that it's probably better than any of the ones mentioned above.
> >
> > The idea is to make `allocable` return the portion of the input resources
> > that are allocatable, and strip the unelectable portion.
> >
> > For example:
> > 1) If the input resources are "cpus:0.001,gpus:1", the `allocatable`
> method
> > will return "gpus:1".
> > 2) If the input resources are "cpus:1,mem:1", the `allocatable` method
> will
> > return "cpus:1".
> > 3) If the input resources are "cpus:0.001,mem:1", the `allocatable`
> method
> > will return an empty Resources object.
> >
> > Basically, the algorithm is like the following:
> >
> > allocatable = input
> > foreach known resource type t: do
> >   r = resources of type t from the input
> >   if r is less than the min resource of type t; then
> > allocatable -= r
> >   fi
> > done
> > return allocatable
> >
> > Let me know what do you guys think!
> >
> > Thanks!
> > - Jie
> >
> > On Fri, Mar 2, 2018 at 4:44 PM, Benjamin Mahler 
> > wrote:
> >
> > > I think (2) would need to be:
> > >
> > > bool HierarchicalAllocatorProcess::allocatable(
> > > const Resources& resources)
> > > {
> > >   if (something outside {cpu, mem, disk} is present) return true
> > >   else return true iff at least one of {cpu, mem, disk} has >=
> {MIN_CPU,
> > > MIN_MEM, MIN_DISK}
> > > }
> > >
> > > Otherwise, 1 GPU would be offered but 1GPU + 0.001 CPU would not?
> > >
> > > On Fri, Mar 2, 2018 at 9:27 AM, Jie Yu  wrote:
> > >
> > > > Hi,
> > > >
> > > > The allocatable
> > > >  > > > ator/mesos/hierarchical.cpp#L2471-L2479>
> > > > check in the allocator (shown below) was originally introduced to
> help
> > > > alleviate the situation where a framework receives some resources,
> but
> > no
> > > > cpu/memory, thus cannot launch a task.
> > > >
> > > > bool HierarchicalAllocatorProcess::allocatable(
> > > > const Resources& resources)
> > > > {
> > > >   Option cpus = resources.cpus();
> > > >   Option mem = resources.mem();
> > > >
> > > >   return (cpus.isSome() && cpus.get() >= MIN_CPUS) ||
> > > >  (mem.isSome() && mem.get() >= MIN_MEM);
> > > > }
> > > >
> > > > As pointed by Benjamin in MESOS-7398
> > > > , it now seems to
> > > mainly
> > > > help to minimize the performance overhead from too many small offers
> > > > (instead too small resource amounts are kept out of the offer pool
> > until
> > > > they became accumulated into larger resources).
> > > >
> > > > This check does cause issues when new resources types are introduced.
> > For
> > > > instance, this check does prevent GPU resources alone from being
> > > allocated
> > > > to a framework. There are some other issues we discover MESOS-8626
> > > > .
> > > >
> > > > There are several proposals:
> > > >
> > > > (1) *Completely remove this check*. This check is a heuristic anyway,
> > and
> > > > only applies to a subset of resources (cpu/memory). However, there
> > might
> > > be
> > > > some implication of that change since it's also leveraged to prevent
> > too
> > > > many small offers. *If you are concerned about this approach, please
> > > raise
> > > > your voice.*
> > > >
> > > > (2) *Consider adjust the check to the following. *
> > > >
> > > > bool HierarchicalAllocatorProcess::allocatable(
> > > > const Resources& resources)
> > > > {
> > > >   Option cpus = resources.cpus();
> > > >   Option mem = resources.mem();
> > > >
> > > >   if 

Re: Reconsidering `allocatable` check in the allocator

[Qian Zhang]

So if the input resources are "cpus:0.001,disk:1024", the `allocatable`
method will return "disk:1024"? This seems not compatible with the existing
behavior: with the current implementation of `allocatable`, the same input
resources will be just skipped because we think "cpus:0.001" is too small
for framework to launch a task.

allocatable = input
> foreach known resource type t: do
>   r = resources of type t from the input
>   if r is less than the min resource of type t; then
> allocatable -= r
>   fi
> done
> return allocatable
>

Are we going to define min amount for each known resource type (including
disk and gpu)?


Regards,
Qian Zhang

On Wed, Mar 7, 2018 at 6:10 AM, Jie Yu  wrote:

> Chatted with BenM offline on this. There's another option what both of us
> agreed that it's probably better than any of the ones mentioned above.
>
> The idea is to make `allocable` return the portion of the input resources
> that are allocatable, and strip the unelectable portion.
>
> For example:
> 1) If the input resources are "cpus:0.001,gpus:1", the `allocatable` method
> will return "gpus:1".
> 2) If the input resources are "cpus:1,mem:1", the `allocatable` method will
> return "cpus:1".
> 3) If the input resources are "cpus:0.001,mem:1", the `allocatable` method
> will return an empty Resources object.
>
> Basically, the algorithm is like the following:
>
> allocatable = input
> foreach known resource type t: do
>   r = resources of type t from the input
>   if r is less than the min resource of type t; then
> allocatable -= r
>   fi
> done
> return allocatable
>
> Let me know what do you guys think!
>
> Thanks!
> - Jie
>
> On Fri, Mar 2, 2018 at 4:44 PM, Benjamin Mahler 
> wrote:
>
> > I think (2) would need to be:
> >
> > bool HierarchicalAllocatorProcess::allocatable(
> > const Resources& resources)
> > {
> >   if (something outside {cpu, mem, disk} is present) return true
> >   else return true iff at least one of {cpu, mem, disk} has >= {MIN_CPU,
> > MIN_MEM, MIN_DISK}
> > }
> >
> > Otherwise, 1 GPU would be offered but 1GPU + 0.001 CPU would not?
> >
> > On Fri, Mar 2, 2018 at 9:27 AM, Jie Yu  wrote:
> >
> > > Hi,
> > >
> > > The allocatable
> > >  > > ator/mesos/hierarchical.cpp#L2471-L2479>
> > > check in the allocator (shown below) was originally introduced to help
> > > alleviate the situation where a framework receives some resources, but
> no
> > > cpu/memory, thus cannot launch a task.
> > >
> > > bool HierarchicalAllocatorProcess::allocatable(
> > > const Resources& resources)
> > > {
> > >   Option cpus = resources.cpus();
> > >   Option mem = resources.mem();
> > >
> > >   return (cpus.isSome() && cpus.get() >= MIN_CPUS) ||
> > >  (mem.isSome() && mem.get() >= MIN_MEM);
> > > }
> > >
> > > As pointed by Benjamin in MESOS-7398
> > > , it now seems to
> > mainly
> > > help to minimize the performance overhead from too many small offers
> > > (instead too small resource amounts are kept out of the offer pool
> until
> > > they became accumulated into larger resources).
> > >
> > > This check does cause issues when new resources types are introduced.
> For
> > > instance, this check does prevent GPU resources alone from being
> > allocated
> > > to a framework. There are some other issues we discover MESOS-8626
> > > .
> > >
> > > There are several proposals:
> > >
> > > (1) *Completely remove this check*. This check is a heuristic anyway,
> and
> > > only applies to a subset of resources (cpu/memory). However, there
> might
> > be
> > > some implication of that change since it's also leveraged to prevent
> too
> > > many small offers. *If you are concerned about this approach, please
> > raise
> > > your voice.*
> > >
> > > (2) *Consider adjust the check to the following. *
> > >
> > > bool HierarchicalAllocatorProcess::allocatable(
> > > const Resources& resources)
> > > {
> > >   Option cpus = resources.cpus();
> > >   Option mem = resources.mem();
> > >
> > >   if (cpus.isSome() && mem.isSome()) {
> > > return cpus.get() >= MIN_CPUS || mem.get() >= MIN_MEM;
> > >   } else if (cpus.isSome()) {
> > > return cpus.get() >= MIN_CPUS;
> > >   } else if (mem.isSome()) {
> > > return mem.get() >= MIN_MEM;
> > >   } else {
> > > return true;
> > >   }
> > > }
> > >
> > > Let me know what you think! Thanks!
> > >
> > > - Jie
> > >
> >
>

Re: Reconsidering `allocatable` check in the allocator

[Jie Yu]

Chatted with BenM offline on this. There's another option what both of us
agreed that it's probably better than any of the ones mentioned above.

The idea is to make `allocable` return the portion of the input resources
that are allocatable, and strip the unelectable portion.

For example:
1) If the input resources are "cpus:0.001,gpus:1", the `allocatable` method
will return "gpus:1".
2) If the input resources are "cpus:1,mem:1", the `allocatable` method will
return "cpus:1".
3) If the input resources are "cpus:0.001,mem:1", the `allocatable` method
will return an empty Resources object.

Basically, the algorithm is like the following:

allocatable = input
foreach known resource type t: do
  r = resources of type t from the input
  if r is less than the min resource of type t; then
allocatable -= r
  fi
done
return allocatable

Let me know what do you guys think!

Thanks!
- Jie

On Fri, Mar 2, 2018 at 4:44 PM, Benjamin Mahler  wrote:

> I think (2) would need to be:
>
> bool HierarchicalAllocatorProcess::allocatable(
> const Resources& resources)
> {
>   if (something outside {cpu, mem, disk} is present) return true
>   else return true iff at least one of {cpu, mem, disk} has >= {MIN_CPU,
> MIN_MEM, MIN_DISK}
> }
>
> Otherwise, 1 GPU would be offered but 1GPU + 0.001 CPU would not?
>
> On Fri, Mar 2, 2018 at 9:27 AM, Jie Yu  wrote:
>
> > Hi,
> >
> > The allocatable
> >  > ator/mesos/hierarchical.cpp#L2471-L2479>
> > check in the allocator (shown below) was originally introduced to help
> > alleviate the situation where a framework receives some resources, but no
> > cpu/memory, thus cannot launch a task.
> >
> > bool HierarchicalAllocatorProcess::allocatable(
> > const Resources& resources)
> > {
> >   Option cpus = resources.cpus();
> >   Option mem = resources.mem();
> >
> >   return (cpus.isSome() && cpus.get() >= MIN_CPUS) ||
> >  (mem.isSome() && mem.get() >= MIN_MEM);
> > }
> >
> > As pointed by Benjamin in MESOS-7398
> > , it now seems to
> mainly
> > help to minimize the performance overhead from too many small offers
> > (instead too small resource amounts are kept out of the offer pool until
> > they became accumulated into larger resources).
> >
> > This check does cause issues when new resources types are introduced. For
> > instance, this check does prevent GPU resources alone from being
> allocated
> > to a framework. There are some other issues we discover MESOS-8626
> > .
> >
> > There are several proposals:
> >
> > (1) *Completely remove this check*. This check is a heuristic anyway, and
> > only applies to a subset of resources (cpu/memory). However, there might
> be
> > some implication of that change since it's also leveraged to prevent too
> > many small offers. *If you are concerned about this approach, please
> raise
> > your voice.*
> >
> > (2) *Consider adjust the check to the following. *
> >
> > bool HierarchicalAllocatorProcess::allocatable(
> > const Resources& resources)
> > {
> >   Option cpus = resources.cpus();
> >   Option mem = resources.mem();
> >
> >   if (cpus.isSome() && mem.isSome()) {
> > return cpus.get() >= MIN_CPUS || mem.get() >= MIN_MEM;
> >   } else if (cpus.isSome()) {
> > return cpus.get() >= MIN_CPUS;
> >   } else if (mem.isSome()) {
> > return mem.get() >= MIN_MEM;
> >   } else {
> > return true;
> >   }
> > }
> >
> > Let me know what you think! Thanks!
> >
> > - Jie
> >
>

Re: Reconsidering `allocatable` check in the allocator

[Benjamin Mahler]

I think (2) would need to be:

bool HierarchicalAllocatorProcess::allocatable(
const Resources& resources)
{
  if (something outside {cpu, mem, disk} is present) return true
  else return true iff at least one of {cpu, mem, disk} has >= {MIN_CPU,
MIN_MEM, MIN_DISK}
}

Otherwise, 1 GPU would be offered but 1GPU + 0.001 CPU would not?

On Fri, Mar 2, 2018 at 9:27 AM, Jie Yu  wrote:

> Hi,
>
> The allocatable
>  ator/mesos/hierarchical.cpp#L2471-L2479>
> check in the allocator (shown below) was originally introduced to help
> alleviate the situation where a framework receives some resources, but no
> cpu/memory, thus cannot launch a task.
>
> bool HierarchicalAllocatorProcess::allocatable(
> const Resources& resources)
> {
>   Option cpus = resources.cpus();
>   Option mem = resources.mem();
>
>   return (cpus.isSome() && cpus.get() >= MIN_CPUS) ||
>  (mem.isSome() && mem.get() >= MIN_MEM);
> }
>
> As pointed by Benjamin in MESOS-7398
> , it now seems to mainly
> help to minimize the performance overhead from too many small offers
> (instead too small resource amounts are kept out of the offer pool until
> they became accumulated into larger resources).
>
> This check does cause issues when new resources types are introduced. For
> instance, this check does prevent GPU resources alone from being allocated
> to a framework. There are some other issues we discover MESOS-8626
> .
>
> There are several proposals:
>
> (1) *Completely remove this check*. This check is a heuristic anyway, and
> only applies to a subset of resources (cpu/memory). However, there might be
> some implication of that change since it's also leveraged to prevent too
> many small offers. *If you are concerned about this approach, please raise
> your voice.*
>
> (2) *Consider adjust the check to the following. *
>
> bool HierarchicalAllocatorProcess::allocatable(
> const Resources& resources)
> {
>   Option cpus = resources.cpus();
>   Option mem = resources.mem();
>
>   if (cpus.isSome() && mem.isSome()) {
> return cpus.get() >= MIN_CPUS || mem.get() >= MIN_MEM;
>   } else if (cpus.isSome()) {
> return cpus.get() >= MIN_CPUS;
>   } else if (mem.isSome()) {
> return mem.get() >= MIN_MEM;
>   } else {
> return true;
>   }
> }
>
> Let me know what you think! Thanks!
>
> - Jie
>

Reconsidering `allocatable` check in the allocator

[Jie Yu]

Hi,

The allocatable

check in the allocator (shown below) was originally introduced to help
alleviate the situation where a framework receives some resources, but no
cpu/memory, thus cannot launch a task.

bool HierarchicalAllocatorProcess::allocatable(
const Resources& resources)
{
  Option cpus = resources.cpus();
  Option mem = resources.mem();

  return (cpus.isSome() && cpus.get() >= MIN_CPUS) ||
 (mem.isSome() && mem.get() >= MIN_MEM);
}

As pointed by Benjamin in MESOS-7398
, it now seems to mainly
help to minimize the performance overhead from too many small offers
(instead too small resource amounts are kept out of the offer pool until
they became accumulated into larger resources).

This check does cause issues when new resources types are introduced. For
instance, this check does prevent GPU resources alone from being allocated
to a framework. There are some other issues we discover MESOS-8626
.

There are several proposals:

(1) *Completely remove this check*. This check is a heuristic anyway, and
only applies to a subset of resources (cpu/memory). However, there might be
some implication of that change since it's also leveraged to prevent too
many small offers. *If you are concerned about this approach, please raise
your voice.*

(2) *Consider adjust the check to the following. *

bool HierarchicalAllocatorProcess::allocatable(
const Resources& resources)
{
  Option cpus = resources.cpus();
  Option mem = resources.mem();

  if (cpus.isSome() && mem.isSome()) {
return cpus.get() >= MIN_CPUS || mem.get() >= MIN_MEM;
  } else if (cpus.isSome()) {
return cpus.get() >= MIN_CPUS;
  } else if (mem.isSome()) {
return mem.get() >= MIN_MEM;
  } else {
return true;
  }
}

Let me know what you think! Thanks!

- Jie