Here's some input :). If throttling is tolerable but preemption is not, how would that be expressed? (Is that supported?) [Klaus]: It's not supported; only revocable resources has this attribute: non-throttleable or throttleable. The throttleable revocable resources is reported by ResourceEstimator which means the resources maybe throttled by its original owner.
How does this work with the QoS controller? Will there be a new correction type to indicate throttling, or does throttling happen "behind the agent's back"? [Klaus]: The QoSController/ResourceEstimator only manages throttleable revocable resources; the others resources (regular resources and non-throttleable revocable resources) are managed by allocator. The "manage" means generation and destroy/eviction. Regarding "throttling happen", good question. I think the throttling will dependent on containers, let me double check it :). If any comments, please let me know. ---- Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer Platform OpenSource Technology, STG, IBM GCG +86-10-8245 4084 | klaus1982...@gmail.com | http://k82.me On Sat, Mar 19, 2016 at 11:15 PM, <connor....@gmail.com> wrote: > Thanks for the good explanations so far Ben and Klaus. Apologies if you > guys already covered these questions in the meeting: > > If throttling is tolerable but preemption is not, how would that be > expressed? (Is that supported?) > > How does this work with the QoS controller? Will there be a new correction > type to indicate throttling, or does throttling happen "behind the agent's > back"? > > Thanks, > -- > Connor > > > On Mar 19, 2016, at 04:01, Klaus Ma <klaus1982...@gmail.com> wrote: > > > > @team, in the latest meeting, we agree to keep current name ThrottleInfo. > > > > If any more comments, please let me know. > > > >> On Wednesday, March 16, 2016 at 9:32:37 PM UTC+8, Guangya Liu wrote: > >> Also please show your comments if any for the name here, the current > name is ThrottleInfo, in Kubernetes resources qos design document, they are > using scavenging as the key work for such behaviour, so a possible name > here could be ScavengeInfo , please show your comments if any for those two > names or even if you want to propose a new name here. > >> > >> message RevocableInfo { > >> message ThrottleInfo {} > >> > >> // If set, indicates that the resources may be throttled at > >> // any time. Throttle-able resoruces can be used for tasks > >> // that do not have strict performance requirements and are > >> // capable of handling being throttled. > >> optional ThrottleInfo throttle_info = 1; > >> } > >> > >> 在 2016年3月16日星期三 UTC+8上午10:24:14，Klaus Ma写道： > >>> > >>> The patches are updated accordingly; JIRA: MESOS-3888 , RR: > https://reviews.apache.org/r/40375/ . > >>> > >>> Thanks > >>> klaus > >>> > >>>> On Saturday, March 12, 2016 at 11:09:46 AM UTC+8, Benjamin Mahler > wrote: > >>>> Hey folks, > >>>> > >>>> In the resource allocation working group we've been looking into a > few projects that will make the allocator able to offer out resources as > revocable. For example: > >>>> > >>>> -We'll want to eventually allocate resources as revocable _by > default_, only allowing non-revocable when there are guarantees put in > place (static reservations or quota). > >>>> > >>>> -On the path to revocable by default, we can incrementally start to > offer certain resources as revocable. Consider when quota is set but the > role isn't using all of the quota. The unallocated quota can be offered to > other roles, but it should be revocable because we may revoke them should > the quota'ed role want to use the resources. Unused reservations fall into > a similar category. > >>>> > >>>> -Going revocable by default also allows us to enforce fairness in a > dynamically changing cluster by revoking resources as weights are changed, > frameworks are added or removed, etc. > >>>> > >>>> In this context, "revocable" means that the resources may be taken > away and the container will be destroyed. The meaning of "revocable" in the > context of usage oversubscription includes this, but also the container may > experience a throttling (e.g. lower cpu shares, less network priority, etc). > >>>> > >>>> For this reason, and because we internally need to distinguish > revocable resources between the those that are generated by usage > oversubscription and those that are generated by the allocator, we're > thinking of the following change to the API: > >>>> > >>>> > >>>> > >>>> - message RevocableInfo {} > >>>> + message RevocableInfo { > >>>> + message ThrottleInfo {} > >>>> + > >>>> + // If set, indicates that the resources may be throttled at > >>>> + // any time. Throttle-able resoruces can be used for tasks > >>>> + // that do not have strict performance requirements and are > >>>> + // capable of handling being throttled. > >>>> + optional ThrottleInfo throttle_info; > >>>> + } > >>>> > >>>> // If this is set, the resources are revocable, i.e., any tasks or > >>>> - // executors launched using these resources could get preempted or > >>>> - // throttled at any time. This could be used by frameworks to run > >>>> - // best effort tasks that do not need strict uptime or performance > >>>> + // executors launched using these resources could be terminated at > >>>> + // any time. This could be used by frameworks to run > >>>> + // best effort tasks that do not need strict uptime > >>>> // guarantees. Note that if this is set, 'disk' or 'reservation' > >>>> // cannot be set. > >>>> optional RevocableInfo revocable = 9; > >>>> > >>>> > >>>> > >>>> Essentially we want to distinguish between revocable and revocable + > throttle-able. This is because usage-oversubscription generates > throttle-able revocable resources, whereas the allocator does not. This > also solves our problem of distinguishing between these two kinds of > revocable resources internally. > >>>> > >>>> Feedback welcome! > >>>> > >>>> Ben > >>>> >
