Totally agree that ultimately a single way of dealing with offers is
wanted. Quota does seem like a way forward although its lack of chunkiness
dilutes to what extent it guarantees progress.
On Wed, Jan 18, 2017 at 1:59 AM Povilas Versockas
wrote:
> Thanks for information and ideas! I think labeled static reservation would
> help if framework could modify labels during runtime.
>
> Personally I think statically reserved resources should be offered only to
> frameworks but look like free (role:*) resources. Then framework developers
> could use labeled dynamic reservations and there won’t be any code
> differences handling static reservations and dynamic reservations. And this
> would make current frameworks like Cassandra or dcos-commons library
> support static reservation.
>
> Given the current situation it looks like quota may be a solution for me.
> The current idea is to create a custom resource on my mesos-agents and set
> a quota.
>
> Example:
>
> Set mesos-agent with flag:
>
> --resources
> cpus(*):8;mem(*):4096;disk(*):4096;ports(*):[31000-32000];mysql(*):1
>
>
> And then using operators API set quota:
>
> {
>
>"role": “role”,
>
>"guarantee": [
>
> {
>
>"name": "cpus",
>
>"type": "SCALAR",
>
>"scalar": { "value": 8 }
>
> },
>
> {
>
>"name": "mem",
>
>"type": "SCALAR",
>
>"scalar": { "value": 4096 }
>
> },
>
> …
>
> {
>
>"name": “mysql”,
>
>"type": "SCALAR",
>
>"scalar": { "value": 1 }
>
> }
>
>]
>
> }
>
>
> This should make mesos-master save agent’s resources for my role and let
> me use dynamic reservations.
>
>
>
> On Wed, Jan 18, 2017 at 2:29 AM, Greg Mann wrote:
>
> Thanks Gabriel, that makes sense. It sounds like labels on static
> reservations might be the most expedient path toward a solution to this
> problem, but that is not without its complications, as suggested in the
> related ticket which Neil filed a while back:
> https://issues.apache.org/jira/browse/MESOS-4476
>
> Povilas, also see this related ticket that Gabriel pointed me to:
> https://issues.apache.org/jira/browse/MESOS-6939
>
> It sounds like this is a real issue for stateful framework developers, so
> hopefully we will find some time soon to implement a solution. In the
> meantime, Povilas, I'm afraid to say I don't know exactly what solution to
> recommend. If anybody else in the community has some ideas, it would be
> great to hear them :)
>
> Cheers,
> Greg
>
>
> On Tue, Jan 17, 2017 at 2:52 PM, Gabriel Hartmann
> wrote:
>
> @Greg: The reason people use static reservation is to enforce that
> particular resources (usually disks) can only be consumed by a particular
> framework. They also don't know when the stateful service is going to be
> installed necessarily so they don't want to race with other frameworks to
> consume those special resources. So static reservation is desirable.
> However, all stateful services also need more information about reserved
> resources than is natively provided by Mesos in the static reservation case
> (i.e. the labels he describes). `dcos-commons` does the same thing.
> Various work arounds exist, but none are able to provide resource
> allocation enforcement because only roles do that. An alternate resource
> allocation enforcement mechanism is needed. Usually this is the part where
> people start talking about quota.
>
> Neither option 1 nor option 2 provided a race proof way to get fully
> labeled reserved resources. It's been proposed in the past that it be
> allowed to add labels to statically reserved resources. That's kind of
> fine except now you have these things that can't really be UNRESERVEd but
> look exactly like dynamic resources which can...
>
> Quota w/ chunks as a step in the deployment of stateful services is very
> desirable in an adversarial environment. However if your'e in a
> cooperative environment (i.e. you're not in an adversarial relationship
> with other frameworks) if you had resources (particularly disk resources)
> with attributes on them you could have frameworks voluntarily choose not to
> consume resources not meant for them.
>
> e.g. Disk resource has attribute `CASSANDRA`. Ok, since I'm a Kafka
> framework I won't go use that disk.
>
> On Tue, Jan 17, 2017 at 11:24 AM Greg Mann wrote:
>
> Hi Povilas,
> Another approach you could try is to use dynamic reservations only. You
> could either:
>
>1. Alter your stateful framework to dynamically reserve the resources
>that it needs, or
>2. Add a script to your cluster tooling that would make use of the
>operator endpoint for dynamic reservations [1]
> to
>dynamically reserve the stateful framework's resources when your cluster is
>initially provisioned. This would have a similar effect to static
>reservations, but would allow you to set labels
>
> Approach #1 makes sense to me; is there a reas
