Ben, when do you have your next mesos allocator sync? We don't have our next performance isolation sync lined up yet, so we could piggy back on yours if you have it scheduled already.
Niklas On Mon, Mar 14, 2016 at 9:32 AM, Jie Yu <[email protected]> wrote: > > > > Just a quick note: Ian D. and the performance isolation working group are > > discussing similar annotations and we should meet and talk about the > > options. > > > +1 > > Would love to understand the relationship between this and the > task/executor level annotations. > > - Jie > > On Mon, Mar 14, 2016 at 9:29 AM, Niklas Nielsen <[email protected]> wrote: > > > Hi Ben, > > > > Just a quick note: Ian D. and the performance isolation working group are > > discussing similar annotations and we should meet and talk about the > > options. > > > > Niklas > > > > On Sat, Mar 12, 2016 at 12:05 AM, Klaus Ma <[email protected]> > wrote: > > > > > Yes, I think that's true for now; so we define `ThrottleInfo` as > message > > to > > > be more flexible. In Optimistic Offer Phase 1, we only use it to > > > distinguish usage oversubscriptions and allocation oversubscription, > > > similar to bool :). > > > > > > Regarding the resources type, two questions after the discussion: > > > > > > 1. should we send different offer to the framework, so when > > > usage/allocation oversubscription updated, only one type of offer will > be > > > rescinded? > > > 2. should we define framework's capability against `ThrottleInfo`? > > > > > > ---- > > > Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer > > > Platform OpenSource Technology, STG, IBM GCG > > > +86-10-8245 4084 | [email protected] | http://k82.me > > > > > > On Sat, Mar 12, 2016 at 12:03 PM, Guangya Liu <[email protected]> > > wrote: > > > > > > > > > > > Hi Ben, > > > > > > > > I think that currently and even in the near future, the > > __ThrottleInfo__ > > > > will only be used by the usage oversubscriptions and the > > oversubscription > > > > for allocator (Both quota and reservations) will not use this value > but > > > > only using __RevocableInfo__ is enough. > > > > > > > > I can even think that the __ThrottleInfo__ as a boolean value in > > > > optimistic offer phase 1 as it is mainly used to distinguish > resources > > > > between usage oversubscriptions and allocation oversubscription > (Quota > > > and > > > > Reservations), comments? > > > > > > > > Thanks, > > > > > > > > Guangya > > > > > > > > 在 2016年3月12日星期六 UTC+8上午11:09:46，Benjamin Mahler写道： > > > > > > > >> Hey folks, > > > >> > > > >> In the resource allocation working group we've been looking into a > few > > > >> projects that will make the allocator able to offer out resources as > > > >> revocable. For example: > > > >> > > > >> -We'll want to eventually allocate resources as revocable _by > > default_, > > > >> only allowing non-revocable when there are guarantees put in place > > > (static > > > >> reservations or quota). > > > >> > > > >> -On the path to revocable by default, we can incrementally start to > > > offer > > > >> certain resources as revocable. Consider when quota is set but the > > role > > > >> isn't using all of the quota. The unallocated quota can be offered > to > > > other > > > >> roles, but it should be revocable because we may revoke them should > > the > > > >> quota'ed role want to use the resources. Unused reservations fall > > into a > > > >> similar category. > > > >> > > > >> -Going revocable by default also allows us to enforce fairness in a > > > >> dynamically changing cluster by revoking resources as weights are > > > changed, > > > >> frameworks are added or removed, etc. > > > >> > > > >> In this context, "revocable" means that the resources may be taken > > away > > > >> and the container will be destroyed. The meaning of "revocable" in > the > > > >> context of usage oversubscription includes this, but also the > > container > > > may > > > >> experience a throttling (e.g. lower cpu shares, less network > priority, > > > etc). > > > >> > > > >> For this reason, and because we internally need to distinguish > > revocable > > > >> resources between the those that are generated by usage > > oversubscription > > > >> and those that are generated by the allocator, we're thinking of the > > > >> following change to the API: > > > >> > > > >> > > > >> > > > >> - message RevocableInfo {} > > > >> + message RevocableInfo { > > > >> + message ThrottleInfo {} > > > >> + > > > >> + // If set, indicates that the resources may be throttled at > > > >> + // any time. Throttle-able resoruces can be used for tasks > > > >> + // that do not have strict performance requirements and are > > > >> + // capable of handling being throttled. > > > >> + optional ThrottleInfo throttle_info; > > > >> + } > > > >> > > > >> // If this is set, the resources are revocable, i.e., any tasks > or > > > >> - // executors launched using these resources could get preempted > or > > > >> - // throttled at any time. This could be used by frameworks to run > > > >> - // best effort tasks that do not need strict uptime or > performance > > > >> + // executors launched using these resources could be terminated > at > > > >> + // any time. This could be used by frameworks to run > > > >> + // best effort tasks that do not need strict uptime > > > >> // guarantees. Note that if this is set, 'disk' or 'reservation' > > > >> // cannot be set. > > > >> optional RevocableInfo revocable = 9; > > > >> > > > >> > > > >> > > > >> Essentially we want to distinguish between revocable and revocable + > > > >> throttle-able. This is because usage-oversubscription generates > > > >> throttle-able revocable resources, whereas the allocator does not. > > This > > > >> also solves our problem of distinguishing between these two kinds of > > > >> revocable resources internally. > > > >> > > > >> Feedback welcome! > > > >> > > > >> Ben > > > >> > > > >> -- > > > > You received this message because you are subscribed to the Google > > Groups > > > > "Mesos Resource Allocation Working Group" group. > > > > To unsubscribe from this group and stop receiving emails from it, > send > > an > > > > email to [email protected]. > > > > To post to this group, send email to > [email protected] > > . > > > > To view this discussion on the web visit > > > > > > > > > > https://groups.google.com/d/msgid/mesos-allocation/a68b9e92-f22e-4cf7-9499-b982c9c07613%40googlegroups.com > > > > < > > > > > > https://groups.google.com/d/msgid/mesos-allocation/a68b9e92-f22e-4cf7-9499-b982c9c07613%40googlegroups.com?utm_medium=email&utm_source=footer > > > > > > > > . > > > > For more options, visit https://groups.google.com/d/optout. > > > > > > > > > > > > > > > -- > > Niklas > > > -- Niklas
