Re: [openstack-dev] [nova] [placement] placement update 18-14
On Fri, 6 Apr 2018, Chris Dent wrote: * Eric and I discussed earlier in the week that it might be a good time to start an #openstack-placement IRC channel, for two main reasons: break things up so as to limit the crosstalk in the often very busy #openstack-nova channel and to lend a bit of momentum for going in that direction. Is this okay with everyone? If not, please say so, otherwise I'll make it happen soon. After confirmation in today's scheduler meeting this has been done. #openstack-placement now exists, is registered, and various *bot additions are in progress: https://review.openstack.org/559768 https://review.openstack.org/559769 http://p.anticdent.org/logs/openstack-placement -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent tw: @anticdent__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [placement] placement update 18-14
On Fri, Apr 6, 2018 at 2:54 PM, Chris Dent wrote: > > This is "contract" style update. New stuff will not be added to the > lists. > > # Most Important > > There doesn't appear to be anything new with regard to most > important. That which was important remains important. At the > scheduler team meeting at the start of the week there was talk of > working out ways to trim the amount of work in progress by using the > nova priorities tracking etherpad to help sort things out: > > https://etherpad.openstack.org/p/rocky-nova-priorities-tracking > > Update provider tree and nested allocation candidates remain > critical basic functionality on which much else is based. With most > of provider tree done, it's really on nested allocation candidates. > > # What's Changed > > Quite a bit of provider tree related code has merged. > > Some negotiation happened with regard to when/if the fixes for > shared providers is going to happen. I'm not sure how that resolved, > if someone can follow up with that, that would be most excellent. > > Most of the placement-req-filter series merged. > > The spec for error codes in the placement API merged (code is in > progress and ready for review, see below). > > # Questions > > * Eric and I discussed earlier in the week that it might be a good > time to start an #openstack-placement IRC channel, for two main > reasons: break things up so as to limit the crosstalk in the often > very busy #openstack-nova channel and to lend a bit of momentum > for going in that direction. Is this okay with everyone? If not, > please say so, otherwise I'll make it happen soon. > > Fine by me. It's sometimes difficult to follow all the conversations so having a separate channel looks good to me, at least for discussing only about specific Placement questions. For Nova related points (like how to use nested RPs for example with NUMA), maybe #openstack-nova is still the main IRC channel for that. * Shared providers status? > (I really think we need to make this go. It was one of the > original value propositions of placement: being able to accurate > manage shared disk.) > > # Bugs > > * Placement related bugs not yet in progress: https://goo.gl/TgiPXb >15, -1 on last week > * In progress placement bugs: https://goo.gl/vzGGDQ >13, +1 on last week > > # Specs > > These seem to be divided into three classes: > > * Normal stuff > * Old stuff not getting attention or newer stuff that ought to be > abandoned because of lack of support > * Anything related to the client side of using nested providers > effectively. This apparently needs a lot of thinking. If there are > some general sticking points we can extract and resolve, that > might help move the whole thing forward? > > * https://review.openstack.org/#/c/549067/ > VMware: place instances on resource pool > (using update_provider_tree) > > * https://review.openstack.org/#/c/545057/ > mirror nova host aggregates to placement API > > * https://review.openstack.org/#/c/552924/ > Proposes NUMA topology with RPs > > * https://review.openstack.org/#/c/544683/ > Account for host agg allocation ratio in placement > > * https://review.openstack.org/#/c/552927/ > Spec for isolating configuration of placement database > (This has a strong +2 on it but needs one more.) > > * https://review.openstack.org/#/c/552105/ > Support default allocation ratios > > * https://review.openstack.org/#/c/438640/ > Spec on preemptible servers > > * https://review.openstack.org/#/c/556873/ >Handle nested providers for allocation candidates > > * https://review.openstack.org/#/c/556971/ >Add Generation to Consumers > > * https://review.openstack.org/#/c/557065/ >Proposes Multiple GPU types > > * https://review.openstack.org/#/c/555081/ >Standardize CPU resource tracking > > * https://review.openstack.org/#/c/502306/ >Network bandwidth resource provider > > * https://review.openstack.org/#/c/509042/ >Propose counting quota usage from placement > > # Main Themes > > ## Update Provider Tree > > Most of the main guts of this have merged (huzzah!). What's left are > some loose end details, and clean handling of aggregates: > > https://review.openstack.org/#/q/topic:bp/update-provider-tree > > ## Nested providers in allocation candidates > > Representing nested provides in the response to GET > /allocation_candidates is required to actually make use of all the > topology that update provider tree will report. That work is in > progress at: > > https://review.openstack.org/#/q/topic:bp/nested-resource-providers > https://review.openstack.org/#/q/topic:bp/nested-resource-pr > oviders-allocation-candidates > > Note that some of this includes the up-for-debate shared handling. > > ## Request Filters > > As far as I can tell this is mostly done (yay!) but there is a loose > end: We merged an updated spec to support multiple member_of > parameters, but it's not clear any
Re: [openstack-dev] [nova] [placement] placement update 18-14
Hi Novaers, On 2018/04/07 6:41, Eric Fried wrote: Some negotiation happened with regard to when/if the fixes for shared providers is going to happen. I'm not sure how that resolved, if someone can follow up with that, that would be most excellent. This is the subject of another thread [2] that's still "dangling". We discussed it in the sched meeting this week [3] and concluded [4] that we shouldn't do it in Rocky. BUT tetsuro later pointed out that part of the series in question [5] is still needed to satisfy NRP-in-alloc-cands (return the whole tree's providers in provider_summaries - even the ones that aren't providing resource to the request). That patch changes behavior, so needs a microversion (mostly done already in that patch), so needs a spec. We haven't yet resolved whether this is truly needed, so haven't assigned a body to the spec work. Specs are where we discuss whether proposed functions are truly needed, so I've uploaded the spec[7] and put my thoughts there :) [7] https://review.openstack.org/#/c/559466/ The implementation is in [8]. I've also submitted on it several patches for nested scenario. [8] https://review.openstack.org/#/c/558045/ [2] http://lists.openstack.org/pipermail/openstack-dev/2018-March/128944.html [3] http://eavesdrop.openstack.org/meetings/nova_scheduler/2018/nova_scheduler.2018-04-02-14.00.log.html#l-91 [4] http://eavesdrop.openstack.org/meetings/nova_scheduler/2018/nova_scheduler.2018-04-02-14.00.log.html#l-137 [5] https://review.openstack.org/#/c/558045/ [6] http://eavesdrop.openstack.org/meetings/nova_scheduler/2018/nova_scheduler.2018-04-02-14.00.log.html#l-104 P.S. The hottest news in Japan this week is Shohei Otani's home runs @Los Angeles Angels. He started playing in MLB this year. You should +2 on this without discussions. Thanks! -- Tetsuro Nakamura NTT Network Service Systems Laboratories TEL:0422 59 6914(National)/+81 422 59 6914(International) 3-9-11, Midori-Cho Musashino-Shi, Tokyo 180-8585 Japan __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [placement] placement update 18-14
>> it's really on nested allocation candidates. > > Yup. And that series is deadlocked on a disagreement about whether > granular request groups should be "separate by default" (meaning: if you > request multiple groups of resources, the expectation is that they will > be served by distinct resource providers) or "unrestricted by default" > (meaning: if you request multiple groups of resources, those resources > may or may not be serviced by distinct resource providers). This is really a granular thing, not a nested thing. I was holding up the nrp-in-alloc-cands spec [1] for other reasons, but I've stopped doing that now. We should be able to proceed with the nrp work. I'm working on the granular code, wherein I can hopefully isolate the separate-vs-unrestricted decision such that we can go either way once that issue is resolved. [1] https://review.openstack.org/#/c/556873/ >> Some negotiation happened with regard to when/if the fixes for >> shared providers is going to happen. I'm not sure how that resolved, >> if someone can follow up with that, that would be most excellent. This is the subject of another thread [2] that's still "dangling". We discussed it in the sched meeting this week [3] and concluded [4] that we shouldn't do it in Rocky. BUT tetsuro later pointed out that part of the series in question [5] is still needed to satisfy NRP-in-alloc-cands (return the whole tree's providers in provider_summaries - even the ones that aren't providing resource to the request). That patch changes behavior, so needs a microversion (mostly done already in that patch), so needs a spec. We haven't yet resolved whether this is truly needed, so haven't assigned a body to the spec work. I believe Jay is still planning [6] to parse and respond to the ML thread. After he clones himself. [2] http://lists.openstack.org/pipermail/openstack-dev/2018-March/128944.html [3] http://eavesdrop.openstack.org/meetings/nova_scheduler/2018/nova_scheduler.2018-04-02-14.00.log.html#l-91 [4] http://eavesdrop.openstack.org/meetings/nova_scheduler/2018/nova_scheduler.2018-04-02-14.00.log.html#l-137 [5] https://review.openstack.org/#/c/558045/ [6] http://eavesdrop.openstack.org/meetings/nova_scheduler/2018/nova_scheduler.2018-04-02-14.00.log.html#l-104 >> * Shared providers status? >> (I really think we need to make this go. It was one of the >> original value propositions of placement: being able to accurate >> manage shared disk.) > > Agreed, but you know NUMA. And CPU pinning. And vGPUs. And FPGAs. > And physnet network bandwidth scheduling. And... well, you get the idea. Right. I will say that Tetsuro has been doing an excellent job slinging code for this, though. So the bottleneck is really reviewer bandwidth (already an issue for the work we *are* trying to fit in Rocky). If it's still on the table by Stein, we ought to consider making it a high priority. (Our Rocky punchlist seems to be favoring "urgent" over "important" to some extent.) -efried __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [placement] placement update 18-14
Thanks, as always, for the excellent summary emails, Chris. Comments inline. On 04/06/2018 01:54 PM, Chris Dent wrote: This is "contract" style update. New stuff will not be added to the lists. # Most Important There doesn't appear to be anything new with regard to most important. That which was important remains important. At the scheduler team meeting at the start of the week there was talk of working out ways to trim the amount of work in progress by using the nova priorities tracking etherpad to help sort things out: https://etherpad.openstack.org/p/rocky-nova-priorities-tracking Update provider tree and nested allocation candidates remain critical basic functionality on which much else is based. With most of provider tree done, it's really on nested allocation candidates. Yup. And that series is deadlocked on a disagreement about whether granular request groups should be "separate by default" (meaning: if you request multiple groups of resources, the expectation is that they will be served by distinct resource providers) or "unrestricted by default" (meaning: if you request multiple groups of resources, those resources may or may not be serviced by distinct resource providers). For folk's information, the latter (unrestricted by default) is the *existing* behaviour as outlined in the granular request groups spec: http://specs.openstack.org/openstack/nova-specs/specs/rocky/approved/granular-resource-requests.html Specifically, it is Requirement 3 on the above spec that is the primary driver for this debate. I currently have an action item to resolve this debate and move forward with a decision, whatever that may be. # What's Changed Quite a bit of provider tree related code has merged. Some negotiation happened with regard to when/if the fixes for shared providers is going to happen. I'm not sure how that resolved, if someone can follow up with that, that would be most excellent. Sharing providers are in a weird place right now, agreed. We have landed lots of code on the placement side of the house for handling sharing providers. However, the nova-compute service still does not know about the providers that share resources with it. This makes it impossible right now to have a compute node with local disk storage as well as shared disk resources. Most of the placement-req-filter series merged. The spec for error codes in the placement API merged (code is in progress and ready for review, see below). # Questions * Eric and I discussed earlier in the week that it might be a good time to start an #openstack-placement IRC channel, for two main reasons: break things up so as to limit the crosstalk in the often very busy #openstack-nova channel and to lend a bit of momentum for going in that direction. Is this okay with everyone? If not, please say so, otherwise I'll make it happen soon. Cool with me. I know Matt has wanted a separate placement channel for a while now. * Shared providers status? (I really think we need to make this go. It was one of the original value propositions of placement: being able to accurate manage shared disk.) Agreed, but you know NUMA. And CPU pinning. And vGPUs. And FPGAs. And physnet network bandwidth scheduling. And... well, you get the idea. Best, -jay # Bugs * Placement related bugs not yet in progress: https://goo.gl/TgiPXb 15, -1 on last week * In progress placement bugs: https://goo.gl/vzGGDQ 13, +1 on last week # Specs These seem to be divided into three classes: * Normal stuff * Old stuff not getting attention or newer stuff that ought to be abandoned because of lack of support * Anything related to the client side of using nested providers effectively. This apparently needs a lot of thinking. If there are some general sticking points we can extract and resolve, that might help move the whole thing forward? * https://review.openstack.org/#/c/549067/ VMware: place instances on resource pool (using update_provider_tree) * https://review.openstack.org/#/c/545057/ mirror nova host aggregates to placement API * https://review.openstack.org/#/c/552924/ Proposes NUMA topology with RPs * https://review.openstack.org/#/c/544683/ Account for host agg allocation ratio in placement * https://review.openstack.org/#/c/552927/ Spec for isolating configuration of placement database (This has a strong +2 on it but needs one more.) * https://review.openstack.org/#/c/552105/ Support default allocation ratios * https://review.openstack.org/#/c/438640/ Spec on preemptible servers * https://review.openstack.org/#/c/556873/ Handle nested providers for allocation candidates * https://review.openstack.org/#/c/556971/ Add Generation to Consumers * https://review.openstack.org/#/c/557065/ Proposes Multiple GPU types * https://review.openstack.org/#/c/555081/ Standardize CPU resource tracking * https://
[openstack-dev] [nova] [placement] placement update 18-14
This is "contract" style update. New stuff will not be added to the lists. # Most Important There doesn't appear to be anything new with regard to most important. That which was important remains important. At the scheduler team meeting at the start of the week there was talk of working out ways to trim the amount of work in progress by using the nova priorities tracking etherpad to help sort things out: https://etherpad.openstack.org/p/rocky-nova-priorities-tracking Update provider tree and nested allocation candidates remain critical basic functionality on which much else is based. With most of provider tree done, it's really on nested allocation candidates. # What's Changed Quite a bit of provider tree related code has merged. Some negotiation happened with regard to when/if the fixes for shared providers is going to happen. I'm not sure how that resolved, if someone can follow up with that, that would be most excellent. Most of the placement-req-filter series merged. The spec for error codes in the placement API merged (code is in progress and ready for review, see below). # Questions * Eric and I discussed earlier in the week that it might be a good time to start an #openstack-placement IRC channel, for two main reasons: break things up so as to limit the crosstalk in the often very busy #openstack-nova channel and to lend a bit of momentum for going in that direction. Is this okay with everyone? If not, please say so, otherwise I'll make it happen soon. * Shared providers status? (I really think we need to make this go. It was one of the original value propositions of placement: being able to accurate manage shared disk.) # Bugs * Placement related bugs not yet in progress: https://goo.gl/TgiPXb 15, -1 on last week * In progress placement bugs: https://goo.gl/vzGGDQ 13, +1 on last week # Specs These seem to be divided into three classes: * Normal stuff * Old stuff not getting attention or newer stuff that ought to be abandoned because of lack of support * Anything related to the client side of using nested providers effectively. This apparently needs a lot of thinking. If there are some general sticking points we can extract and resolve, that might help move the whole thing forward? * https://review.openstack.org/#/c/549067/ VMware: place instances on resource pool (using update_provider_tree) * https://review.openstack.org/#/c/545057/ mirror nova host aggregates to placement API * https://review.openstack.org/#/c/552924/ Proposes NUMA topology with RPs * https://review.openstack.org/#/c/544683/ Account for host agg allocation ratio in placement * https://review.openstack.org/#/c/552927/ Spec for isolating configuration of placement database (This has a strong +2 on it but needs one more.) * https://review.openstack.org/#/c/552105/ Support default allocation ratios * https://review.openstack.org/#/c/438640/ Spec on preemptible servers * https://review.openstack.org/#/c/556873/ Handle nested providers for allocation candidates * https://review.openstack.org/#/c/556971/ Add Generation to Consumers * https://review.openstack.org/#/c/557065/ Proposes Multiple GPU types * https://review.openstack.org/#/c/555081/ Standardize CPU resource tracking * https://review.openstack.org/#/c/502306/ Network bandwidth resource provider * https://review.openstack.org/#/c/509042/ Propose counting quota usage from placement # Main Themes ## Update Provider Tree Most of the main guts of this have merged (huzzah!). What's left are some loose end details, and clean handling of aggregates: https://review.openstack.org/#/q/topic:bp/update-provider-tree ## Nested providers in allocation candidates Representing nested provides in the response to GET /allocation_candidates is required to actually make use of all the topology that update provider tree will report. That work is in progress at: https://review.openstack.org/#/q/topic:bp/nested-resource-providers https://review.openstack.org/#/q/topic:bp/nested-resource-providers-allocation-candidates Note that some of this includes the up-for-debate shared handling. ## Request Filters As far as I can tell this is mostly done (yay!) but there is a loose end: We merged an updated spec to support multiple member_of parameters, but it's not clear anybody is currently owning that: https://review.openstack.org/#/c/555413/ ## Mirror nova host aggregates to placement This makes it so some kinds of aggregate filtering can be done "placement side" by mirroring nova host aggregates into placement aggregates. https://review.openstack.org/#/q/topic:bp/placement-mirror-host-aggregates It's part of what will make the req filters above useful. ## Forbidden Traits A way of expressing "I'd like resources that do _not_ have trait X". This is ready for review: https://review.openstack.org/#/q/topic:bp/placement-forbidden-