Re: [openstack-dev] [nova] Reflections on the pike-1 milestone

2017-04-20 Thread Maish Saidel-Keesing
Perfect explanation and very clear - Thanks Matt!!


On 20/04/17 21:18, Matt Riedemann wrote:
> On 4/20/2017 2:01 AM, Maish Saidel-Keesing wrote:
>> Just as a matter of interest - from the numbers above you say 62
>> blueprints approved - was this only for this cycle - or *up until* this
>> cycle.
>>
>> When you mention several up for review - can you elaborate on exact
>> numbers?
>>
>> I am not looking to 'monitor' activity - but for me it would be
>> interesting to understand - what the workload is actually like. If the
>> ratio of 'incoming work' (blueprints) vs. completed/in-review is 62:3
>> then to me - this seems to be something that needs to be addressed.
>>
>> Or am I misunderstanding the comment above?
>
> That means 62 blueprints approved for Pike (this cycle). This does not
> mean the code is merged and the blueprint is completed. It means we
> agreed on the design proposal and can move forward with code for the
> blueprint.
>
> Several up for review means there are approved blueprints with code
> ready for review (they have started, or are more than just a POC).
> These numbers are probably low right now, but all blueprints targeted
> for Pike [1] with Delivery status of "Needs Code Review" is what I'm
> referring to here, which is currently 38.
>
> As for incoming work, and the ratio you point out, is not unusual in
> the first milestone before we do our spec freeze, where we then stop
> accepting new blueprint proposals for the release so we can focus on
> implementing and reviewing what we've already planned to do.
>
> If you want a much more detailed explanation of the numbers and
> trends, I provided that after the Ocata release [2].
>
> [1] https://blueprints.launchpad.net/nova/pike
> [2]
> http://lists.openstack.org/pipermail/openstack-dev/2017-February/111639.html
>

-- 
Best Regards,
Maish Saidel-Keesing

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Reflections on the pike-1 milestone

2017-04-20 Thread Matt Riedemann

On 4/20/2017 2:01 AM, Maish Saidel-Keesing wrote:

Just as a matter of interest - from the numbers above you say 62
blueprints approved - was this only for this cycle - or *up until* this
cycle.

When you mention several up for review - can you elaborate on exact
numbers?

I am not looking to 'monitor' activity - but for me it would be
interesting to understand - what the workload is actually like. If the
ratio of 'incoming work' (blueprints) vs. completed/in-review is 62:3
then to me - this seems to be something that needs to be addressed.

Or am I misunderstanding the comment above?


That means 62 blueprints approved for Pike (this cycle). This does not 
mean the code is merged and the blueprint is completed. It means we 
agreed on the design proposal and can move forward with code for the 
blueprint.


Several up for review means there are approved blueprints with code 
ready for review (they have started, or are more than just a POC). These 
numbers are probably low right now, but all blueprints targeted for Pike 
[1] with Delivery status of "Needs Code Review" is what I'm referring to 
here, which is currently 38.


As for incoming work, and the ratio you point out, is not unusual in the 
first milestone before we do our spec freeze, where we then stop 
accepting new blueprint proposals for the release so we can focus on 
implementing and reviewing what we've already planned to do.


If you want a much more detailed explanation of the numbers and trends, 
I provided that after the Ocata release [2].


[1] https://blueprints.launchpad.net/nova/pike
[2] 
http://lists.openstack.org/pipermail/openstack-dev/2017-February/111639.html


--

Thanks,

Matt

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Reflections on the pike-1 milestone

2017-04-20 Thread Maish Saidel-Keesing
On 20/04/17 0:42, Matt Riedemann wrote:
> Hey everyone,
>
> Now that the pike-1 milestone is behind us I wanted to have a recap of
> the milestone to compare what progress we made against goals we set at
> the PTG, and to look forward to the pike-2 milestone.
>
> First some highlights of things accomplished in the pike-1 milestone
> in no particular order:
>
> - Jay Pipes got the Ironic virt driver reporting custom resource
> classes into the Placement service for compute node inventory.
> - There is good progress on the os-traits library and Alex Xu got the
> /traits API merged into the placement endpoint.
> - Sean Dague got high-level agreement on unifying limits in Keystone
> which is a foundation for supporting hierarchical quotas.
> - We merged the spec and plan for integrating Searchlight into
> nova-api. At this point that's all just spec, but it was a pretty
> complicated spec to work through and we have a plan going into pike-2.
> - Sean Dague got uwsgi working in devstack now and Chris Dent is
> working on making nova-api run under uwsgi per the Pike community goal.
> - Dan Smith has made good progress on enabling multi-cell support in
> the REST API and getting devstack to run and pass tests with a fleet
> of conductors. We'll be discussing this at the Forum [1].
> - We merged Ildiko Vancsa's patch to remove the check_attach code from
> Nova, and we merged John Garbutt's spec for integrating the new Cinder
> attachment APIs into Nova. Progress has been made on the code for
> using the new APIs too.
> - Chris Dent has been sending weekly emails giving updates on the work
> going on with placement, and Balazs Gibizer has been doing similar for
> the versioned notifications work. This has been helpful for keeping
> focus, recording decisions, and giving those outside the day-to-day
> involvement an idea of the progress made and where they can help.
> - Good progress from the OSIC team on documenting the various policy
> rules [2].
> - We have 62 blueprints/specs approved, 3 completed, and several with
> code up for review.
>
Just as a matter of interest - from the numbers above you say 62
blueprints approved - was this only for this cycle - or *up until* this
cycle.

When you mention several up for review - can you elaborate on exact
numbers?

I am not looking to 'monitor' activity - but for me it would be
interesting to understand - what the workload is actually like. If the
ratio of 'incoming work' (blueprints) vs. completed/in-review is 62:3
then to me - this seems to be something that needs to be addressed.

Or am I misunderstanding the comment above?
> Some targets we missed in pike-1:
>
> - We aren't as far along as we'd like to be with the counting quotas
> work, but to be fair, some of that was redone after initial review to
> make it easier to integrate. And we did approve the spec for putting a
> /usages API into placement which the quotas work will leverage.
> - We don't have the additional-notification-fields-for-searchlight
> blueprint done yet. We hit some snags during review but those have
> been ironed out now, so we should be able to finish this early in pike-2.
> - We never had a spec for using Cinder as an ephemeral backend.
> However, we will be discussing this at the Forum [3] so hopefully
> we'll have a plan going into Queens.
> - The versioned notifications transformation has been slowing down,
> probably due to a lack of reviews.
> - I never delivered a spec for deprecating personality files from the
> compute REST API (but I'm deprecating some other things from the API,
> so that counts, right?).
> - We didn't merge a spec to support the concept of service-locked
> instances. There is a draft work in progress spec though to pick up in
> Queens [4].
> - Little to no progress on merging the network-aware scheduling series
> which has been carried over since Newton. This is needed to support
> Neutron routed networks.
> - The PowerVM driver series has not landed a single change yet due to
> lack of reviews.
>
> Looking to pike-2 we have a few priority things to get done:
>
> - Get a dsvm job running with nova + searchlight and start writing the
> proof of concept for searchlight integration with nova-api. The goal
> here is going to be finding out what issues we didn't anticipate in
> the spec, even though there were plenty of issues already identified
> in the spec. We will also be discussing this at the forum [5].
> - Complete the additional-notification-fields-for-searchlight blueprint.
> - We need to make progress on landing the counting quotas changes
> early so we can shake out any bugs introduced by that complicated change.
> - Close on the plan for moving claims to the scheduler, discuss it
> with operators at the Forum [6], and make good progress on
> implementation by the end of the milestone.
> - Get more of the versioned notifications work done.
> - Now that the /traits API is available, we need to make progress on
> adding support for modeling shared storage 

[openstack-dev] [nova] Reflections on the pike-1 milestone

2017-04-19 Thread Matt Riedemann

Hey everyone,

Now that the pike-1 milestone is behind us I wanted to have a recap of 
the milestone to compare what progress we made against goals we set at 
the PTG, and to look forward to the pike-2 milestone.


First some highlights of things accomplished in the pike-1 milestone in 
no particular order:


- Jay Pipes got the Ironic virt driver reporting custom resource classes 
into the Placement service for compute node inventory.
- There is good progress on the os-traits library and Alex Xu got the 
/traits API merged into the placement endpoint.
- Sean Dague got high-level agreement on unifying limits in Keystone 
which is a foundation for supporting hierarchical quotas.
- We merged the spec and plan for integrating Searchlight into nova-api. 
At this point that's all just spec, but it was a pretty complicated spec 
to work through and we have a plan going into pike-2.
- Sean Dague got uwsgi working in devstack now and Chris Dent is working 
on making nova-api run under uwsgi per the Pike community goal.
- Dan Smith has made good progress on enabling multi-cell support in the 
REST API and getting devstack to run and pass tests with a fleet of 
conductors. We'll be discussing this at the Forum [1].
- We merged Ildiko Vancsa's patch to remove the check_attach code from 
Nova, and we merged John Garbutt's spec for integrating the new Cinder 
attachment APIs into Nova. Progress has been made on the code for using 
the new APIs too.
- Chris Dent has been sending weekly emails giving updates on the work 
going on with placement, and Balazs Gibizer has been doing similar for 
the versioned notifications work. This has been helpful for keeping 
focus, recording decisions, and giving those outside the day-to-day 
involvement an idea of the progress made and where they can help.
- Good progress from the OSIC team on documenting the various policy 
rules [2].
- We have 62 blueprints/specs approved, 3 completed, and several with 
code up for review.


Some targets we missed in pike-1:

- We aren't as far along as we'd like to be with the counting quotas 
work, but to be fair, some of that was redone after initial review to 
make it easier to integrate. And we did approve the spec for putting a 
/usages API into placement which the quotas work will leverage.
- We don't have the additional-notification-fields-for-searchlight 
blueprint done yet. We hit some snags during review but those have been 
ironed out now, so we should be able to finish this early in pike-2.
- We never had a spec for using Cinder as an ephemeral backend. However, 
we will be discussing this at the Forum [3] so hopefully we'll have a 
plan going into Queens.
- The versioned notifications transformation has been slowing down, 
probably due to a lack of reviews.
- I never delivered a spec for deprecating personality files from the 
compute REST API (but I'm deprecating some other things from the API, so 
that counts, right?).
- We didn't merge a spec to support the concept of service-locked 
instances. There is a draft work in progress spec though to pick up in 
Queens [4].
- Little to no progress on merging the network-aware scheduling series 
which has been carried over since Newton. This is needed to support 
Neutron routed networks.
- The PowerVM driver series has not landed a single change yet due to 
lack of reviews.


Looking to pike-2 we have a few priority things to get done:

- Get a dsvm job running with nova + searchlight and start writing the 
proof of concept for searchlight integration with nova-api. The goal 
here is going to be finding out what issues we didn't anticipate in the 
spec, even though there were plenty of issues already identified in the 
spec. We will also be discussing this at the forum [5].

- Complete the additional-notification-fields-for-searchlight blueprint.
- We need to make progress on landing the counting quotas changes early 
so we can shake out any bugs introduced by that complicated change.
- Close on the plan for moving claims to the scheduler, discuss it with 
operators at the Forum [6], and make good progress on implementation by 
the end of the milestone.

- Get more of the versioned notifications work done.
- Now that the /traits API is available, we need to make progress on 
adding support for modeling shared storage pools in Placement.
- Have a multi-cell CI job running which tests the conductor fleet 
deployment model and API, including move (migrate) operations within a cell.
- Continue adding support for the new Cinder attachment APIs. We should 
have the code in place to create new-style attachments by the end of 
pike-2, and testing it with the grenade upgrade CI job. This is needed 
for supporting volume multi-attach.
- Get some of the PowerVM driver patches landed, at least through 
spawn/destroy, but ideally to the point of supporting a console.


Current focus:

- We have the summit coming up in less than three weeks. People are 
working on presentations and planning for the Forum