Re: [openstack-dev] [nova] Review runways this cycle

[melanie witt]

On Tue, 20 Mar 2018 16:44:57 -0700, Melanie Witt wrote:

As mentioned in the earlier "Rocky PTG summary - miscellaneous topics
from Friday" email, this cycle we're going to experiment with a
"runways" system for focusing review on approved blueprints in
time-boxes. The goal here is to use a bit more structure and process in
order to focus review and complete merging of approved work more quickly
and reliably.

We were thinking of starting the runways process after the spec review
freeze (which is April 19) so that reviewers won't be split between spec
reviews and reviews of work in runways.

The process and instructions are explained in detail on this etherpad,
which will also serve as the place we queue and track blueprints for
runways:

https://etherpad.openstack.org/p/nova-runways-rocky

Please bear with us as this is highly experimental and we will be giving
it a go knowing it's imperfect and adjusting the process iteratively as
we learn from it.


Okay, based on the responses on the discussion in the last nova meeting 
and this ML thread, I think we have consensus to go ahead and start 
using runways next week after the spec review day, so:


  Spec review day: Tuesday March 27

  Start using runways: Wednesday March 28

Please add your blueprints to the Queue if the requirements explained on 
the etherpad are met. And please ask questions, in #openstack-nova or on 
this thread, if you have any questions about the process.


We will be moving spec freeze out to r-2 (June 7) to lessen pressure on 
spec review while runways are underway, to get more time to review the 
current queue of approved implementations via runways, and to give 
ourselves the chance to approve more specs along the way if we find 
we're reducing the queue enough by completing blueprints.


Thanks,
-melanie





__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Review runways this cycle

[melanie witt]

On Thu, 22 Mar 2018 16:12:47 -0500, Matt Riedemann wrote:

On 3/22/2018 2:59 PM, melanie witt wrote:

And (MHO) I'm not sure we need help in reviewing more specs.


I wholly disagree here. If you're on the core team, or want to be on the
core team, you should be reviewing specs, because those are the things
that lay out the high level design and thinking about what eventually
comes out in the code.

If there are core team members that aren't involved in the specs review
process, I certainly hope they are going back to do their homework on
the agreed-to design *before* digging into code review.

There are some specs that are pretty simple/mechanical changes, but
there are others that take quite a bit of time ironing out details and
edge cases, and sometimes changes in the initial design, such that it's
important to have that context in mind when you're reviewing the code.

There have been plenty of times I've gone through a lengthy spec review
process and then during implementation review I find things and say,
"wait, in the spec we said...". If you're not involved in both, you're
likely to miss those things.

At the least it gets the context in your head so you're not starting
from scratch.

Maybe you were just saying, "we don't need to review more specs because
we already have enough approved specs to get through the related code
changes", and that's fair, I've said the same before, but those are two
different things.


Yes, the last paragraph is what I meant. I said that in response to the 
idea of putting unapproved specs into the runways right now to get 
review on them. I was saying we probably already have so many approved 
specs that we're at risk of not being able to review and merge all of 
the related code in time. (At this moment there are 41 blueprints 
approved for Rocky). So I'm not sure we need to be getting more focus on 
reviewing more specs so we can approve more of them than we already have 
at this point, IMHO.


-melanie





__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Review runways this cycle

[Matt Riedemann]

On 3/22/2018 2:59 PM, melanie witt wrote:

And (MHO) I'm not sure we need help in reviewing more specs.


I wholly disagree here. If you're on the core team, or want to be on the 
core team, you should be reviewing specs, because those are the things 
that lay out the high level design and thinking about what eventually 
comes out in the code.


If there are core team members that aren't involved in the specs review 
process, I certainly hope they are going back to do their homework on 
the agreed-to design *before* digging into code review.


There are some specs that are pretty simple/mechanical changes, but 
there are others that take quite a bit of time ironing out details and 
edge cases, and sometimes changes in the initial design, such that it's 
important to have that context in mind when you're reviewing the code.


There have been plenty of times I've gone through a lengthy spec review 
process and then during implementation review I find things and say, 
"wait, in the spec we said...". If you're not involved in both, you're 
likely to miss those things.


At the least it gets the context in your head so you're not starting 
from scratch.


Maybe you were just saying, "we don't need to review more specs because 
we already have enough approved specs to get through the related code 
changes", and that's fair, I've said the same before, but those are two 
different things.


--

Thanks,

Matt

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Review runways this cycle

[Eric Fried]

WFM.

November Oscar Victor Alpha you are cleared for takeoff.

On 03/22/2018 03:18 PM, Matt Riedemann wrote:
> On 3/22/2018 2:59 PM, melanie witt wrote:
>> Maybe a good compromise would be to start runways now and move spec
>> freeze out to r-2 (Jun 7). That way we have less pressure on spec
>> review earlier on, more time to review the current queue of approved
>> implementations via runways, and a chance to approve more specs along
>> the way if we find we're flushing the queue down enough.
> 
> This is what I'd prefer to see.
> 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Review runways this cycle

[Matt Riedemann]

On 3/22/2018 2:59 PM, melanie witt wrote:
Maybe a good compromise would be to start runways now and move spec 
freeze out to r-2 (Jun 7). That way we have less pressure on spec review 
earlier on, more time to review the current queue of approved 
implementations via runways, and a chance to approve more specs along 
the way if we find we're flushing the queue down enough.


This is what I'd prefer to see.

--

Thanks,

Matt

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Review runways this cycle

[melanie witt]

On Thu, 22 Mar 2018 13:26:48 -0500, Eric Fried wrote:



I think the only concern around moving spec freeze out would be that I
thought the original purpose of the spec freeze was to set expectations
early about what was approved and not approved instead of having folks
potentially in the situation where it's technically "maybe" for a large
chunk of the cycle. I'm not sure which most people prefer -- would you
rather know early and definitively whether your blueprint is
approved/not approved or would you rather have the opportunity to get
approval during a larger window in the cycle and not know definitively
early on? Can anyone else chime in here?


This is a fair point.

Putting specs into runways doesn't imply (re)moving spec freeze IMO.
It's just a way to get us using runways RIGHT NOW, so that folks with
ready specs can get reviewed sooner, know whether they're approved
sooner, write their code sooner, and get their *code* into an earlier
runway.

A spec in a runway would be treated like anything else: reviewers focus
on it and the author needs to be available to respond quickly to feedback.

I would expect the ratio of specs:code in runways to start off high and
dwindle rapidly as we approach spec freeze.


This would seem to have the same effect as waiting until after spec 
freeze to start using runways for focusing on reviews of 
implementations. And (MHO) I'm not sure we need help in reviewing more 
specs. While it's true that not a lot of people review specs, we do 
still also repeatedly approve more specs than we can complete in a 
cycle, every cycle.


I'd rather keep things simpler and not add spec reviews into the mix at 
this time.


Maybe a good compromise would be to start runways now and move spec 
freeze out to r-2 (Jun 7). That way we have less pressure on spec review 
earlier on, more time to review the current queue of approved 
implementations via runways, and a chance to approve more specs along 
the way if we find we're flushing the queue down enough.


What does everyone think about that?


It's worth pointing out that there's not an expectation for people to
work more/harder when runways are in play.  Just that it increases the
chances of more people looking at the same things at the same time; and
allows us to bring focus to things that might otherwise languish in
ignoreland.


Yes, this. The goal IMHO is to make an improvement over what we usually 
do, which is having our review efforts scattered across various approved 
implementations. If we could focus a bit and increase the chances that 
we're reviewing the same things at the same time, I think we might have 
a better completion percentage on approved blueprints in the cycle.


-melanie













__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Review runways this cycle

[Eric Fried]

> I think the only concern around moving spec freeze out would be that I
> thought the original purpose of the spec freeze was to set expectations
> early about what was approved and not approved instead of having folks
> potentially in the situation where it's technically "maybe" for a large
> chunk of the cycle. I'm not sure which most people prefer -- would you
> rather know early and definitively whether your blueprint is
> approved/not approved or would you rather have the opportunity to get
> approval during a larger window in the cycle and not know definitively
> early on? Can anyone else chime in here?

This is a fair point.

Putting specs into runways doesn't imply (re)moving spec freeze IMO.
It's just a way to get us using runways RIGHT NOW, so that folks with
ready specs can get reviewed sooner, know whether they're approved
sooner, write their code sooner, and get their *code* into an earlier
runway.

A spec in a runway would be treated like anything else: reviewers focus
on it and the author needs to be available to respond quickly to feedback.

I would expect the ratio of specs:code in runways to start off high and
dwindle rapidly as we approach spec freeze.

It's worth pointing out that there's not an expectation for people to
work more/harder when runways are in play.  Just that it increases the
chances of more people looking at the same things at the same time; and
allows us to bring focus to things that might otherwise languish in
ignoreland.

efried

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Review runways this cycle

[melanie witt]

On Thu, 22 Mar 2018 15:40:20 +, John Garbutt wrote:
On 20 March 2018 at 23:44, melanie witt > wrote:


We were thinking of starting the runways process after the spec
review freeze (which is April 19) so that reviewers won't be split
between spec reviews and reviews of work in runways.


I think spec reviews, blueprint reviews, and code review topics could 
all get a runway slot.


What if we had these queues:
Backlog Queue, Blueprint Runway, Approved Queue, Code Runway

Currently all approved blueprints would sit in the Approved queue.
As described, you leave the runway and go back in the queue if progress 
stalls.


That might be a bit more complex than what I was thinking for our first 
go at trying this process. Is "Backlog Queue" the list of unapproved 
blueprints (specs) waiting for dedicated runway review and "Blueprint 
Runway" are the specs guaranteed to receive review during the two week 
time-box?


I hadn't been thinking about putting spec reviews into runways as well. 
I had been focusing on only the review of implementations of approved specs.



Basically abandon the spec freeze. Control with runways instead.


I'm open to this idea, just had a concern as described in my last reply 
to this thread, on whether getting rid of spec freeze may remove early 
expectation setting for people (if they prefer that). I'm okay with 
going with the majority vote about it.


-melanie




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Review runways this cycle

[melanie witt]

On Thu, 22 Mar 2018 10:38:38 -0500, Matt Riedemann wrote:

On 3/20/2018 6:44 PM, melanie witt wrote:

We were thinking of starting the runways process after the spec review
freeze (which is April 19) so that reviewers won't be split between spec
reviews and reviews of work in runways.


I'm going to try and reign in the other thread [1] and bring it back here.

The request to discuss this in the nova meeting was discussed, log
starts here [2].

For me personally, I'm OK with starting runways now, before the spec
freeze, if we also agree to move the spec freeze out past milestone 1,
so either milestone 2 or feature freeze (~milestone 3).

This is because we have far fewer people that actually review specs, and
I personally don't want to have the expectation/pressure to be
reviewing, at a high level anyway, both specs before the deadline along
with runways at the same time.

Moving out the spec freeze also means that assuming runways works and
we're good about flushing things through the queue, we can seed more
blueprints based on specs we approve later.

Given we already have 33 approved but not yet completed blueprints, we
have plenty of content to keep us busy with runways for the next couple
of months.

If we start runways now and don't move out the spec freeze, I'm OK with
that but I personally will probably be devoting more of my time to
reviewing specs than stuff in the runways.


I kind of like the idea of moving spec freeze out to near milestone 3, 
that is, for the most part not have a freeze, and start runways now.


My thinking is, as was mentioned in the other thread, we've already 
approved ~75% of the number of blueprints we approved last cycle and 
last cycle we had a 79% completion percentage of approved blueprints. 
I'd be inclined to mostly stop approving new things now until we've 
completed some via runways.


I think the only concern around moving spec freeze out would be that I 
thought the original purpose of the spec freeze was to set expectations 
early about what was approved and not approved instead of having folks 
potentially in the situation where it's technically "maybe" for a large 
chunk of the cycle. I'm not sure which most people prefer -- would you 
rather know early and definitively whether your blueprint is 
approved/not approved or would you rather have the opportunity to get 
approval during a larger window in the cycle and not know definitively 
early on? Can anyone else chime in here?


-melanie





__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Review runways this cycle

[John Garbutt]

Hi

So I am really excited for us to try runways out, ASAP.

On 20 March 2018 at 23:44, melanie witt  wrote:

> We were thinking of starting the runways process after the spec review
> freeze (which is April 19) so that reviewers won't be split between spec
> reviews and reviews of work in runways.
>

I think spec reviews, blueprint reviews, and code review topics could all
get a runway slot.

What if we had these queues:
Backlog Queue, Blueprint Runway, Approved Queue, Code Runway

Currently all approved blueprints would sit in the Approved queue.
As described, you leave the runway and go back in the queue if progress
stalls.

Basically abandon the spec freeze. Control with runways instead.

The process and instructions are explained in detail on this etherpad,
> which will also serve as the place we queue and track blueprints for
> runways:
>
> https://etherpad.openstack.org/p/nova-runways-rocky


I like its simplicity.
If progress stalls you are booted out of a run way slot back to the queue.

Having said all that, I am basically happy with anything that gets us
trying this out ASAP.

Thanks,
johnthetubaguy
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Review runways this cycle

[Matt Riedemann]

On 3/20/2018 6:44 PM, melanie witt wrote:
We were thinking of starting the runways process after the spec review 
freeze (which is April 19) so that reviewers won't be split between spec 
reviews and reviews of work in runways.


I'm going to try and reign in the other thread [1] and bring it back here.

The request to discuss this in the nova meeting was discussed, log 
starts here [2].


For me personally, I'm OK with starting runways now, before the spec 
freeze, if we also agree to move the spec freeze out past milestone 1, 
so either milestone 2 or feature freeze (~milestone 3).


This is because we have far fewer people that actually review specs, and 
I personally don't want to have the expectation/pressure to be 
reviewing, at a high level anyway, both specs before the deadline along 
with runways at the same time.


Moving out the spec freeze also means that assuming runways works and 
we're good about flushing things through the queue, we can seed more 
blueprints based on specs we approve later.


Given we already have 33 approved but not yet completed blueprints, we 
have plenty of content to keep us busy with runways for the next couple 
of months.


If we start runways now and don't move out the spec freeze, I'm OK with 
that but I personally will probably be devoting more of my time to 
reviewing specs than stuff in the runways.


[1] 
http://lists.openstack.org/pipermail/openstack-dev/2018-March/128603.html
[2] 
http://eavesdrop.openstack.org/meetings/nova/2018/nova.2018-03-22-14.00.log.html#l-205


--

Thanks,

Matt

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova] Review runways this cycle

[melanie witt]

Hello Stackers,

As mentioned in the earlier "Rocky PTG summary - miscellaneous topics 
from Friday" email, this cycle we're going to experiment with a 
"runways" system for focusing review on approved blueprints in 
time-boxes. The goal here is to use a bit more structure and process in 
order to focus review and complete merging of approved work more quickly 
and reliably.


We were thinking of starting the runways process after the spec review 
freeze (which is April 19) so that reviewers won't be split between spec 
reviews and reviews of work in runways.


The process and instructions are explained in detail on this etherpad, 
which will also serve as the place we queue and track blueprints for 
runways:


https://etherpad.openstack.org/p/nova-runways-rocky

Please bear with us as this is highly experimental and we will be giving 
it a go knowing it's imperfect and adjusting the process iteratively as 
we learn from it.


Do check out the etherpad and ask questions on this thread or on IRC and 
we'll do our best to answer them.


Cheers,
-melanie



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev