Hi all,
We had a nice discussion about this in the API working group meeting today.
I agree that it's a good idea to do our best to make this change compatible
with future updates to the Request call and/or quota. I think it would be
beneficial to have a meeting in a few days to brainstorm some ideas; please
let me know if you would like to be included in that meeting and I will add
you to an invite!
Cheers,
Greg
On Tue, Jun 12, 2018 at 8:06 AM, Alex Rukletsov <a...@mesosphere.com> wrote:
> Instead of the master flag, why not a master API call. This will allow to
> update the value without restarting the master.
>
> Another thought is that we should explain operators how and when to use
> this knob. For example, if they observe a behavioural pattern A, then it
> means B is happening, and tuning the knob to C might help.
>
> On Tue, Jun 12, 2018 at 7:36 AM, Jie Yu <yujie....@gmail.com> wrote:
>
>> I would suggest we also consider the possibility of adding per framework
>> control on `min_allocatable_resources`.
>>
>> If we want to consider supporting per-framework setting, we should
>> probably
>> model this as a protobuf, rather than a free form JSON. The same protobuf
>> can be reused for both master flag, framework API, or even supporting
>> Resource Request in the future. Something like the following:
>>
>> message ResourceQuantityPredicate {
>> enum Type {
>> SCALAR_GE,
>> }
>> optional Type type;
>> optional Value.Scalar scalar;
>> }
>> message ResourceRequirement {
>> required string resource_name;
>> oneof predicates {
>> ResourceQuantityPredicate quantity;
>> }
>> }
>> message ResourceRequirementList {
>> // All requirements MUST be met.
>> repeated ResourceRequirement requirements;
>> }
>>
>> // Resource request API.
>> message Request {
>> repeated ResoruceRequrementList accepted;
>> }
>>
>> // `allocatable()`
>> message MinimalAllocatableResources {
>> repeated ResoruceRequrementList accepted;
>> }
>>
>> On Mon, Jun 11, 2018 at 3:47 PM, Meng Zhu <m...@mesosphere.com> wrote:
>>
>> > Hi:
>> >
>> > The allocatable
>> > <https://github.com/apache/mesos/blob/1.5.x/src/master/alloc
>> ator/mesos/hierarchical.cpp#L2471-L2479>
>> > check in the allocator (shown below) was originally introduced to
>> >
>> > help alleviate the situation where a framework receives some resources,
>> > but no
>> >
>> > cpu/memory, thus cannot launch a task.
>> >
>> >
>> > constexpr double MIN_CPUS = 0.01;constexpr Bytes MIN_MEM =
>> Megabytes(32);
>> > bool HierarchicalAllocatorProcess::allocatable(
>> > const Resources& resources)
>> > {
>> > Option<double> cpus = resources.cpus();
>> > Option<Bytes> mem = resources.mem();
>> >
>> > return (cpus.isSome() && cpus.get() >= MIN_CPUS) ||
>> > (mem.isSome() && mem.get() >= MIN_MEM);
>> > }
>> >
>> >
>> > Issues
>> >
>> > However, there has been a couple of issues surfacing lately surrounding
>> > the check.
>> >
>> > -
>> > - - MESOS-8935 Quota limit "chopping" can lead to cpu-only and
>>
>> > memory-only offers.
>> >
>> > We introduced fined-grained quota-allocation (MESOS-7099) in Mesos 1.5.
>> > When we
>> >
>> > allocate resources to a role, we'll "chop" the available resources of
>> the
>> > agent up to the
>> >
>> > quota limit for the role. However, this has the unintended consequence
>> of
>> > creating
>> >
>> > cpu-only and memory-only offers, even though there might be other agents
>> > with both
>> >
>> > cpu and memory resources available in the cluster.
>> >
>> >
>> > - MESOS-8626 The 'allocatable' check in the allocator is problematic
>> with
>> > multi-role frameworks.
>> >
>> > Consider roleA reserved cpu/memory on an agent and roleB reserved disk
>> on
>> > the same agent.
>> >
>> > A framework under both roleA and roleB will not be able to get the
>> > reserved disk due to the
>> >
>> > allocatable check. With the introduction of resource providers, the
>> > similar situation will
>> >
>> > become more common.
>> >
>> > Proposed change
>> >
>> > Instead of hardcoding a one-size-fits-all value in Mesos, we are
>> proposing
>> > to add a new master flag
>> >
>> > min_allocatable_resources. It specifies one or more scalar resources
>> > quantities that define the
>> >
>> > minimum allocatable resources for the allocator. The allocator will only
>> > offer resources that are more
>> >
>> > than at least one of the specified resources. The default behavior *is
>> > backward compatible* i.e.
>> >
>> > by default, the flag is set to “cpus:0.01|mem:32”.
>> >
>> > Usage
>> >
>> > The flag takes in either a simple text of resource(s) delimited by a bar
>> > (|) or a JSON array of JSON
>> >
>> > formatted resources. Note, the input should be “pure” scalar quantities
>> > i.e. the specified resource(s)
>> >
>> > should only have name, type (set to scalar) and scalar fields set.
>> >
>> >
>> > Examples:
>> >
>> > - - To eliminate cpu or memory only offer due to the quota chopping,
>> > - we could set the flag to “cpus:0.01;mem:32”
>> > -
>> > - - To enable offering disk only offer, we could set the flag to
>> > “disk:32”
>> > -
>> > - - For both, we could set the flag to “cpus:0.01;mem:32|disk:32”.
>> > - Then the allocator will only offer resources that at least contain
>> > “cpus:0.01;mem:32”
>> > - OR resources that at least contain “disk:32”.
>> >
>> >
>> > Let me know what you think! Thanks!
>> >
>> >
>> > -Meng
>> >
>> >
>>
>
>
