Re: Reconsidering `allocatable` check in the allocator

To register our possible use case which also requires relaxing this check: We are looking at whether it's feasible to use a custom resource type to present additional tasks in a group. This means our scheduler could add a TaskGroup with no cpu/mem to an existing executor. Because 1) new TaskGroup

Re: Reconsidering `allocatable` check in the allocator

+1 about it not being about network traffic. I the direction we want to head towards is to express and enforce a minimum granularity for scalar resources. For example: CPU: 0.001, if we say that we can only deal with milli-cpus. Disk: 1, if we say that we can only deal with the MB level of disk

Re: Reconsidering `allocatable` check in the allocator

> On Mar 7, 2018, at 5:52 AM, Benjamin Bannier > wrote: > > Hi, > >> Chatted with BenM offline on this. There's another option what both of us >> agreed that it's probably better than any of the ones mentioned above. >> >> The idea is to make `allocable`

Re: Reconsidering `allocatable` check in the allocator

Hi, > Chatted with BenM offline on this. There's another option what both of us > agreed that it's probably better than any of the ones mentioned above. > > The idea is to make `allocable` return the portion of the input resources > that are allocatable, and strip the unelectable portion. > >

Re: Reconsidering `allocatable` check in the allocator

If we are about to offer some of the resources from a particular agent, why would we filter anything at all? I doubt we should be concerned about the size of the offer representation travelling through the network. If available resources are "cpus:0.001,gpus:1" and we want to allocate GPU, what is

Re: Reconsidering `allocatable` check in the allocator

So if the input resources are "cpus:0.001,disk:1024", the `allocatable` method will return "disk:1024"? This seems not compatible with the existing behavior: with the current implementation of `allocatable`, the same input resources will be just skipped because we think "cpus:0.001" is too small

Re: Reconsidering `allocatable` check in the allocator

Chatted with BenM offline on this. There's another option what both of us agreed that it's probably better than any of the ones mentioned above. The idea is to make `allocable` return the portion of the input resources that are allocatable, and strip the unelectable portion. For example: 1) If

Re: Reconsidering `allocatable` check in the allocator

I think (2) would need to be: bool HierarchicalAllocatorProcess::allocatable( const Resources& resources) { if (something outside {cpu, mem, disk} is present) return true else return true iff at least one of {cpu, mem, disk} has >= {MIN_CPU, MIN_MEM, MIN_DISK} } Otherwise, 1 GPU would be

Reconsidering `allocatable` check in the allocator

Hi, The allocatable check in the allocator (shown below) was originally introduced to help alleviate the situation where a framework receives some resources, but no cpu/memory, thus cannot launch