Re: [openstack-dev] [nova] [placement] placement update 18-14

[Chris Dent]

On Fri, 6 Apr 2018, Chris Dent wrote:


* Eric and I discussed earlier in the week that it might be a good
 time to start an #openstack-placement IRC channel, for two main
 reasons: break things up so as to limit the crosstalk in the often
 very busy #openstack-nova channel and to lend a bit of momentum
 for going in that direction. Is this okay with everyone? If not,
 please say so, otherwise I'll make it happen soon.


After confirmation in today's scheduler meeting this has been done.
#openstack-placement now exists, is registered, and various *bot
additions are in progress:

https://review.openstack.org/559768
https://review.openstack.org/559769
http://p.anticdent.org/logs/openstack-placement


--
Chris Dent   ٩◔̯◔۶   https://anticdent.org/
freenode: cdent tw: @anticdent__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [placement] placement update 18-14

[Sylvain Bauza]

On Fri, Apr 6, 2018 at 2:54 PM, Chris Dent  wrote:

>
> This is "contract" style update. New stuff will not be added to the
> lists.
>
> # Most Important
>
> There doesn't appear to be anything new with regard to most
> important. That which was important remains important. At the
> scheduler team meeting at the start of the week there was talk of
> working out ways to trim the amount of work in progress by using the
> nova priorities tracking etherpad to help sort things out:
>
> https://etherpad.openstack.org/p/rocky-nova-priorities-tracking
>
> Update provider tree and nested allocation candidates remain
> critical basic functionality on which much else is based. With most
> of provider tree done, it's really on nested allocation candidates.
>
> # What's Changed
>
> Quite a bit of provider tree related code has merged.
>
> Some negotiation happened with regard to when/if the fixes for
> shared providers is going to happen. I'm not sure how that resolved,
> if someone can follow up with that, that would be most excellent.
>
> Most of the placement-req-filter series merged.
>
> The spec for error codes in the placement API merged (code is in
> progress and ready for review, see below).
>
> # Questions
>
> * Eric and I discussed earlier in the week that it might be a good
>   time to start an #openstack-placement IRC channel, for two main
>   reasons: break things up so as to limit the crosstalk in the often
>   very busy #openstack-nova channel and to lend a bit of momentum
>   for going in that direction. Is this okay with everyone? If not,
>   please say so, otherwise I'll make it happen soon.
>
>
Fine by me. It's sometimes difficult to follow all the conversations so
having a separate channel looks good to me, at least for discussing only
about specific Placement questions.
For Nova related points (like how to use nested RPs for example with NUMA),
maybe #openstack-nova is still the main IRC channel for that.


* Shared providers status?
>   (I really think we need to make this go. It was one of the
>   original value propositions of placement: being able to accurate
>   manage shared disk.)
>
> # Bugs
>
> * Placement related bugs not yet in progress:  https://goo.gl/TgiPXb
>15, -1 on last week
> * In progress placement bugs: https://goo.gl/vzGGDQ
>13, +1 on last week
>
> # Specs
>
> These seem to be divided into three classes:
>
> * Normal stuff
> * Old stuff not getting attention or newer stuff that ought to be
>   abandoned because of lack of support
> * Anything related to the client side of using nested providers
>   effectively. This apparently needs a lot of thinking. If there are
>   some general sticking points we can extract and resolve, that
>   might help move the whole thing forward?
>
> * https://review.openstack.org/#/c/549067/
>   VMware: place instances on resource pool
>   (using update_provider_tree)
>
> * https://review.openstack.org/#/c/545057/
>   mirror nova host aggregates to placement API
>
> * https://review.openstack.org/#/c/552924/
>  Proposes NUMA topology with RPs
>
> * https://review.openstack.org/#/c/544683/
>  Account for host agg allocation ratio in placement
>
> * https://review.openstack.org/#/c/552927/
>  Spec for isolating configuration of placement database
>  (This has a strong +2 on it but needs one more.)
>
> * https://review.openstack.org/#/c/552105/
>  Support default allocation ratios
>
> * https://review.openstack.org/#/c/438640/
>  Spec on preemptible servers
>
> * https://review.openstack.org/#/c/556873/
>Handle nested providers for allocation candidates
>
> * https://review.openstack.org/#/c/556971/
>Add Generation to Consumers
>
> * https://review.openstack.org/#/c/557065/
>Proposes Multiple GPU types
>
> * https://review.openstack.org/#/c/555081/
>Standardize CPU resource tracking
>
> * https://review.openstack.org/#/c/502306/
>Network bandwidth resource provider
>
> * https://review.openstack.org/#/c/509042/
>Propose counting quota usage from placement
>
> # Main Themes
>
> ## Update Provider Tree
>
> Most of the main guts of this have merged (huzzah!). What's left are
> some loose end details, and clean handling of aggregates:
>
> https://review.openstack.org/#/q/topic:bp/update-provider-tree
>
> ## Nested providers in allocation candidates
>
> Representing nested provides in the response to GET
> /allocation_candidates is required to actually make use of all the
> topology that update provider tree will report. That work is in
> progress at:
>
> https://review.openstack.org/#/q/topic:bp/nested-resource-providers
> https://review.openstack.org/#/q/topic:bp/nested-resource-pr
> oviders-allocation-candidates
>
> Note that some of this includes the up-for-debate shared handling.
>
> ## Request Filters
>
> As far as I can tell this is mostly done (yay!) but there is a loose
> end: We merged an updated spec to support multiple member_of
> 

Re: [openstack-dev] [nova] [placement] placement update 18-14

[TETSURO NAKAMURA]

Hi Novaers,

On 2018/04/07 6:41, Eric Fried wrote:

Some negotiation happened with regard to when/if the fixes for
shared providers is going to happen. I'm not sure how that resolved,
if someone can follow up with that, that would be most excellent.


This is the subject of another thread [2] that's still "dangling".  We
discussed it in the sched meeting this week [3] and concluded [4] that
we shouldn't do it in Rocky.  BUT tetsuro later pointed out that part of
the series in question [5] is still needed to satisfy NRP-in-alloc-cands
(return the whole tree's providers in provider_summaries - even the ones
that aren't providing resource to the request).  That patch changes
behavior, so needs a microversion (mostly done already in that patch),
so needs a spec.  We haven't yet resolved whether this is truly needed,
so haven't assigned a body to the spec work.


Specs are where we discuss whether proposed functions are truly needed, 
so I've uploaded the spec[7] and put my thoughts there :)


[7] https://review.openstack.org/#/c/559466/

The implementation is in [8]. I've also submitted on it several patches 
for nested scenario.


[8] https://review.openstack.org/#/c/558045/



[2]
http://lists.openstack.org/pipermail/openstack-dev/2018-March/128944.html
[3]
http://eavesdrop.openstack.org/meetings/nova_scheduler/2018/nova_scheduler.2018-04-02-14.00.log.html#l-91
[4]
http://eavesdrop.openstack.org/meetings/nova_scheduler/2018/nova_scheduler.2018-04-02-14.00.log.html#l-137
[5] https://review.openstack.org/#/c/558045/
[6]
http://eavesdrop.openstack.org/meetings/nova_scheduler/2018/nova_scheduler.2018-04-02-14.00.log.html#l-104


P.S.
The hottest news in Japan this week is Shohei Otani's home runs @Los 
Angeles Angels. He started playing in MLB this year.

You should +2 on this without discussions.

Thanks!

--
Tetsuro Nakamura 
NTT Network Service Systems Laboratories
TEL:0422 59 6914(National)/+81 422 59 6914(International)
3-9-11, Midori-Cho Musashino-Shi, Tokyo 180-8585 Japan



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [placement] placement update 18-14

[Eric Fried]

>> it's really on nested allocation candidates.
> 
> Yup. And that series is deadlocked on a disagreement about whether
> granular request groups should be "separate by default" (meaning: if you
> request multiple groups of resources, the expectation is that they will
> be served by distinct resource providers) or "unrestricted by default"
> (meaning: if you request multiple groups of resources, those resources
> may or may not be serviced by distinct resource providers).

This is really a granular thing, not a nested thing.  I was holding up
the nrp-in-alloc-cands spec [1] for other reasons, but I've stopped
doing that now.  We should be able to proceed with the nrp work.  I'm
working on the granular code, wherein I can hopefully isolate the
separate-vs-unrestricted decision such that we can go either way once
that issue is resolved.

[1] https://review.openstack.org/#/c/556873/

>> Some negotiation happened with regard to when/if the fixes for
>> shared providers is going to happen. I'm not sure how that resolved,
>> if someone can follow up with that, that would be most excellent.

This is the subject of another thread [2] that's still "dangling".  We
discussed it in the sched meeting this week [3] and concluded [4] that
we shouldn't do it in Rocky.  BUT tetsuro later pointed out that part of
the series in question [5] is still needed to satisfy NRP-in-alloc-cands
(return the whole tree's providers in provider_summaries - even the ones
that aren't providing resource to the request).  That patch changes
behavior, so needs a microversion (mostly done already in that patch),
so needs a spec.  We haven't yet resolved whether this is truly needed,
so haven't assigned a body to the spec work.  I believe Jay is still
planning [6] to parse and respond to the ML thread.  After he clones
himself.

[2]
http://lists.openstack.org/pipermail/openstack-dev/2018-March/128944.html
[3]
http://eavesdrop.openstack.org/meetings/nova_scheduler/2018/nova_scheduler.2018-04-02-14.00.log.html#l-91
[4]
http://eavesdrop.openstack.org/meetings/nova_scheduler/2018/nova_scheduler.2018-04-02-14.00.log.html#l-137
[5] https://review.openstack.org/#/c/558045/
[6]
http://eavesdrop.openstack.org/meetings/nova_scheduler/2018/nova_scheduler.2018-04-02-14.00.log.html#l-104

>> * Shared providers status?
>>    (I really think we need to make this go. It was one of the
>>    original value propositions of placement: being able to accurate
>>    manage shared disk.)
> 
> Agreed, but you know NUMA. And CPU pinning. And vGPUs. And FPGAs.
> And physnet network bandwidth scheduling. And... well, you get the idea.

Right.  I will say that Tetsuro has been doing an excellent job slinging
code for this, though.  So the bottleneck is really reviewer bandwidth
(already an issue for the work we *are* trying to fit in Rocky).

If it's still on the table by Stein, we ought to consider making it a
high priority.  (Our Rocky punchlist seems to be favoring "urgent" over
"important" to some extent.)

-efried

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [placement] placement update 18-14

[Jay Pipes]

Thanks, as always, for the excellent summary emails, Chris. Comments inline.

On 04/06/2018 01:54 PM, Chris Dent wrote:


This is "contract" style update. New stuff will not be added to the
lists.

# Most Important

There doesn't appear to be anything new with regard to most
important. That which was important remains important. At the
scheduler team meeting at the start of the week there was talk of
working out ways to trim the amount of work in progress by using the
nova priorities tracking etherpad to help sort things out:

     https://etherpad.openstack.org/p/rocky-nova-priorities-tracking

Update provider tree and nested allocation candidates remain
critical basic functionality on which much else is based. With most
of provider tree done, it's really on nested allocation candidates.


Yup. And that series is deadlocked on a disagreement about whether 
granular request groups should be "separate by default" (meaning: if you 
request multiple groups of resources, the expectation is that they will 
be served by distinct resource providers) or "unrestricted by default" 
(meaning: if you request multiple groups of resources, those resources 
may or may not be serviced by distinct resource providers).


For folk's information, the latter (unrestricted by default) is the 
*existing* behaviour as outlined in the granular request groups spec:


http://specs.openstack.org/openstack/nova-specs/specs/rocky/approved/granular-resource-requests.html

Specifically, it is Requirement 3 on the above spec that is the primary 
driver for this debate.


I currently have an action item to resolve this debate and move forward 
with a decision, whatever that may be.



# What's Changed

Quite a bit of provider tree related code has merged.

Some negotiation happened with regard to when/if the fixes for
shared providers is going to happen. I'm not sure how that resolved,
if someone can follow up with that, that would be most excellent.


Sharing providers are in a weird place right now, agreed. We have landed 
lots of code on the placement side of the house for handling sharing 
providers. However, the nova-compute service still does not know about 
the providers that share resources with it. This makes it impossible 
right now to have a compute node with local disk storage as well as 
shared disk resources.



Most of the placement-req-filter series merged.

The spec for error codes in the placement API merged (code is in
progress and ready for review, see below).

# Questions

* Eric and I discussed earlier in the week that it might be a good
   time to start an #openstack-placement IRC channel, for two main
   reasons: break things up so as to limit the crosstalk in the often
   very busy #openstack-nova channel and to lend a bit of momentum
   for going in that direction. Is this okay with everyone? If not,
   please say so, otherwise I'll make it happen soon.


Cool with me. I know Matt has wanted a separate placement channel for a 
while now.



* Shared providers status?
   (I really think we need to make this go. It was one of the
   original value propositions of placement: being able to accurate
   manage shared disk.)


Agreed, but you know NUMA. And CPU pinning. And vGPUs. And FPGAs. 
And physnet network bandwidth scheduling. And... well, you get the idea.


Best,
-jay


# Bugs

* Placement related bugs not yet in progress:  https://goo.gl/TgiPXb
    15, -1 on last week
* In progress placement bugs: https://goo.gl/vzGGDQ
    13, +1 on last week

# Specs

These seem to be divided into three classes:

* Normal stuff
* Old stuff not getting attention or newer stuff that ought to be
   abandoned because of lack of support
* Anything related to the client side of using nested providers
   effectively. This apparently needs a lot of thinking. If there are
   some general sticking points we can extract and resolve, that
   might help move the whole thing forward?

* https://review.openstack.org/#/c/549067/
   VMware: place instances on resource pool
   (using update_provider_tree)

* https://review.openstack.org/#/c/545057/
   mirror nova host aggregates to placement API

* https://review.openstack.org/#/c/552924/
  Proposes NUMA topology with RPs

* https://review.openstack.org/#/c/544683/
  Account for host agg allocation ratio in placement

* https://review.openstack.org/#/c/552927/
  Spec for isolating configuration of placement database
  (This has a strong +2 on it but needs one more.)

* https://review.openstack.org/#/c/552105/
  Support default allocation ratios

* https://review.openstack.org/#/c/438640/
  Spec on preemptible servers

* https://review.openstack.org/#/c/556873/
    Handle nested providers for allocation candidates

* https://review.openstack.org/#/c/556971/
    Add Generation to Consumers

* https://review.openstack.org/#/c/557065/
    Proposes Multiple GPU types

* https://review.openstack.org/#/c/555081/
    Standardize CPU resource tracking

*