Re: [openstack-dev] [Neutron] Specs approved for Juno-3 and exceptions

[Mandeep Dhami]

Thanks Jay. I agree with your position on it, and that is exactly what I
would expect as the process in a collaborative community. That "feels like
the right way" ;-)

Unfortunately, there have been situations where we have had to ask a
reviewer multiple times to re-review the code (after issues identified in a
previous review have been addressed). Then you struggle between "am I
pestering the reviewer" vs. "what more can we do/needs to be done, please
help us understand" - and absence of that feedback leads to discouragement
for new contributors and piling up of patches for the big deluge near the
cut-off deadlines. My suggestion was to deal with outliers like that.

If there was a clear guideline, that facilitated the smooth flow of patches
and an automated reminder that did not make the person asking for reviews
feel that he/she is pestering, that might help. Or maybe if we update infra
to report on avg. number of days that a negative review was not re-reviewed
after a new patch, we could just address the outliers when we see them.
Just an idea to address the outliers, not the normal flow.

Regards,
Mandeep
-



On Sat, Jul 26, 2014 at 10:19 AM, Jay Pipes  wrote:

> On 07/25/2014 05:48 PM, Mandeep Dhami wrote:
>
>> Thanks for the deck Jay, that is very helpful.
>>
>> Also, would it help the process by having some clear
>> guidelines/expectations around review time as well? In particular, if
>> you have put a -1 or -2, and the issues that you have identified have
>> been addressed by an update (or at least the original author thinks that
>> he has addressed your concern), is it reasonable to expect that you will
>> re-review in a "reasonable time"? This way, the updates can either
>> proceed, or be rejected, as they are being developed instead of
>> accumulating in a backlog that we then try to get approved on the last
>> day of the cut-off?
>>
>
> Guilty as charged, Mandeep. :( If I have failed to re-review something in
> a timely manner, please don't hesitate to either find me on IRC or send me
> an email saying "hey, don't forget about XYZ". People get behind on reviews
> and sometimes things slip the mind. A gentle reminder is all that is
> needed, usually.
>
> As for a hard number of days before sending an email notification, that
> might be possible, but it's not like we all have our vacation reminders
> linked in to Gerrit ;) I think a more personal email or IRC request for
> specific reviews is more appropriate.
>
> Best,
> -jay
>
>  On Fri, Jul 25, 2014 at 12:50 PM, Steve Gordon > > wrote:
>>
>> - Original Message -
>>  > From: "Jay Pipes" mailto:jaypi...@gmail.com>>
>>  > To: openstack-dev@lists.openstack.org
>> 
>>  >
>>  > On 07/24/2014 10:05 AM, CARVER, PAUL wrote:
>>  > > Alan Kavanagh wrote:
>>  > >
>>  > >> If we have more work being put on the table, then more Core
>>  > >> members would definitely go a long way with assisting this, we
>> cant
>>  > >> wait for folks to be reviewing stuff as an excuse to not get
>>  > >> features landed in a given release.
>>  >
>>  > We absolutely can and should wait for folks to be reviewing stuff
>>  > properly. A large number of problems in OpenStack code and flawed
>> design
>>  > can be attributed to impatience and pushing through code that
>> wasn't ready.
>>  >
>>  > I've said this many times, but the best way to get core reviews on
>>  > patches that you submit is to put the effort into reviewing others'
>>  > code. Core reviewers are more willing to do reviews for someone
>> who is
>>  > clearly trying to help the project in more ways than just pushing
>> their
>>  > own code. Note that, Alan, I'm not trying to imply that you are
>> guilty
>>  > of the above! :) I'm just recommending techniques for the general
>>  > contributor community who are not on a core team (including
>> myself!).
>>
>> I agree with all of the above, I do think however there is another
>> un-addressed area where there *may* be room for optimization - which
>> is how we use the earlier milestones. I apologize in advance because
>> this is somewhat tangential to Alan's points but I think it is
>> relevant to the general frustration around what did/didn't get
>> approved in time for the deadline and ultimately what will or wont
>> get reviewed in time to make the release versus being punted to Kilo
>> or even further down the road.
>>
>> We land very, very, little in terms of feature work in the *-1 and
>> *-2 milestones in each release (and this is not just a Neutron
>> thing). Even though we know without a doubt that the amount of work
>> currently approved for J-3 is not realistic we also know that we
>> will land significantly more features in this milestone than the
>> other two that have already been and gone, which

Re: [openstack-dev] [Neutron] Specs approved for Juno-3 and exceptions

[Jay Pipes]

On 07/25/2014 05:48 PM, Mandeep Dhami wrote:

Thanks for the deck Jay, that is very helpful.

Also, would it help the process by having some clear
guidelines/expectations around review time as well? In particular, if
you have put a -1 or -2, and the issues that you have identified have
been addressed by an update (or at least the original author thinks that
he has addressed your concern), is it reasonable to expect that you will
re-review in a "reasonable time"? This way, the updates can either
proceed, or be rejected, as they are being developed instead of
accumulating in a backlog that we then try to get approved on the last
day of the cut-off?


Guilty as charged, Mandeep. :( If I have failed to re-review something 
in a timely manner, please don't hesitate to either find me on IRC or 
send me an email saying "hey, don't forget about XYZ". People get behind 
on reviews and sometimes things slip the mind. A gentle reminder is all 
that is needed, usually.


As for a hard number of days before sending an email notification, that 
might be possible, but it's not like we all have our vacation reminders 
linked in to Gerrit ;) I think a more personal email or IRC request for 
specific reviews is more appropriate.


Best,
-jay


On Fri, Jul 25, 2014 at 12:50 PM, Steve Gordon mailto:sgor...@redhat.com>> wrote:

- Original Message -
 > From: "Jay Pipes" mailto:jaypi...@gmail.com>>
 > To: openstack-dev@lists.openstack.org

 >
 > On 07/24/2014 10:05 AM, CARVER, PAUL wrote:
 > > Alan Kavanagh wrote:
 > >
 > >> If we have more work being put on the table, then more Core
 > >> members would definitely go a long way with assisting this, we
cant
 > >> wait for folks to be reviewing stuff as an excuse to not get
 > >> features landed in a given release.
 >
 > We absolutely can and should wait for folks to be reviewing stuff
 > properly. A large number of problems in OpenStack code and flawed
design
 > can be attributed to impatience and pushing through code that
wasn't ready.
 >
 > I've said this many times, but the best way to get core reviews on
 > patches that you submit is to put the effort into reviewing others'
 > code. Core reviewers are more willing to do reviews for someone
who is
 > clearly trying to help the project in more ways than just pushing
their
 > own code. Note that, Alan, I'm not trying to imply that you are
guilty
 > of the above! :) I'm just recommending techniques for the general
 > contributor community who are not on a core team (including myself!).

I agree with all of the above, I do think however there is another
un-addressed area where there *may* be room for optimization - which
is how we use the earlier milestones. I apologize in advance because
this is somewhat tangential to Alan's points but I think it is
relevant to the general frustration around what did/didn't get
approved in time for the deadline and ultimately what will or wont
get reviewed in time to make the release versus being punted to Kilo
or even further down the road.

We land very, very, little in terms of feature work in the *-1 and
*-2 milestones in each release (and this is not just a Neutron
thing). Even though we know without a doubt that the amount of work
currently approved for J-3 is not realistic we also know that we
will land significantly more features in this milestone than the
other two that have already been and gone, which to my way of
thinking is actually kind of backwards to the ideal situation.

What is unclear to me however is how much of this is a result of
difficulty identifying and approving less controversial/more
straightforward specifications quickly following summit (keeping in
mind this time around there was arguably some additional delay as
the *-specs repository approach was bedded down), an unavoidable
result of human nature being to *really* push when there is a *hard*
deadline to beat, or just that these earlier milestones are somewhat
impacted from fatigue from the summit (I know a lot of people also
try to take some well earned time off around this period + of course
many are still concentrated on stabilization of the previous
release). As a result it's unclear whether there is anything
concrete that can be done to change this but I thought I would bring
it up in case anyone else has any bright ideas!

 > [SNIP]

 > > We ought to (in my personal opinion) be supplying core reviewers to
 > > at least a couple of OpenStack projects. But one way or another we
 > > need to get more capabilities reviewed and merged. My personal top
 > > disappointments are with the current state of IPv6, HA, and
QoS, but
 > > I'm sure other folks can list lots of other capabilities that
 > > they're re

Re: [openstack-dev] [Neutron] Specs approved for Juno-3 and exceptions

[Ivar Lazzaro]

I agree that it's important to set a guideline for this topic.
What if the said reviewer is "on vacation or indisposed"? Should a fallback
strategy exist for that case? A reviewer could indicate a "delegate core"
to review its -2s whenever he has no chance to do it.

Thanks,
Ivar.


On Fri, Jul 25, 2014 at 5:35 PM, Mandeep Dhami 
wrote:

>
> What would be a good guideline for "timely manner"? I would recommend
> something like 2-3 days unless the reviewer is on vacation or is
> indisposed. Is it possible to update gerrit/jenkins to send reminders to
> reviewers in such a scenario?
>
> Regards,
> Mandeep
> -
>
>
>
>
> On Fri, Jul 25, 2014 at 3:14 PM, Kyle Mestery  wrote:
>
>> On Fri, Jul 25, 2014 at 4:48 PM, Mandeep Dhami 
>> wrote:
>> >
>> > Thanks for the deck Jay, that is very helpful.
>> >
>> > Also, would it help the process by having some clear
>> guidelines/expectations
>> > around review time as well? In particular, if you have put a -1 or -2,
>> and
>> > the issues that you have identified have been addressed by an update
>> (or at
>> > least the original author thinks that he has addressed your concern),
>> is it
>> > reasonable to expect that you will re-review in a "reasonable time"?
>> This
>> > way, the updates can either proceed, or be rejected, as they are being
>> > developed instead of accumulating in a backlog that we then try to get
>> > approved on the last day of the cut-off?
>> >
>> I agree, if someone puts a -2 on a patch stressing an issue and the
>> committer has resolved those issues, the -2 should also be resolved in
>> a timely manner. If the issue can't be resolved in the review itself,
>> as this wiki page [1] indicates, the issue should be moved to the
>> mailing list.
>>
>> Thanks,
>> Kyle
>>
>> [1] https://wiki.openstack.org/wiki/CodeReviewGuidelines
>>
>> > Regards,
>> > Mandeep
>> >
>> >
>> >
>> > On Fri, Jul 25, 2014 at 12:50 PM, Steve Gordon 
>> wrote:
>> >>
>> >> - Original Message -
>> >> > From: "Jay Pipes" 
>> >> > To: openstack-dev@lists.openstack.org
>> >> >
>> >> > On 07/24/2014 10:05 AM, CARVER, PAUL wrote:
>> >> > > Alan Kavanagh wrote:
>> >> > >
>> >> > >> If we have more work being put on the table, then more Core
>> >> > >> members would definitely go a long way with assisting this, we
>> cant
>> >> > >> wait for folks to be reviewing stuff as an excuse to not get
>> >> > >> features landed in a given release.
>> >> >
>> >> > We absolutely can and should wait for folks to be reviewing stuff
>> >> > properly. A large number of problems in OpenStack code and flawed
>> design
>> >> > can be attributed to impatience and pushing through code that wasn't
>> >> > ready.
>> >> >
>> >> > I've said this many times, but the best way to get core reviews on
>> >> > patches that you submit is to put the effort into reviewing others'
>> >> > code. Core reviewers are more willing to do reviews for someone who
>> is
>> >> > clearly trying to help the project in more ways than just pushing
>> their
>> >> > own code. Note that, Alan, I'm not trying to imply that you are
>> guilty
>> >> > of the above! :) I'm just recommending techniques for the general
>> >> > contributor community who are not on a core team (including myself!).
>> >>
>> >> I agree with all of the above, I do think however there is another
>> >> un-addressed area where there *may* be room for optimization - which
>> is how
>> >> we use the earlier milestones. I apologize in advance because this is
>> >> somewhat tangential to Alan's points but I think it is relevant to the
>> >> general frustration around what did/didn't get approved in time for the
>> >> deadline and ultimately what will or wont get reviewed in time to make
>> the
>> >> release versus being punted to Kilo or even further down the road.
>> >>
>> >> We land very, very, little in terms of feature work in the *-1 and *-2
>> >> milestones in each release (and this is not just a Neutron thing). Even
>> >> though we know without a doubt that the amount of work currently
>> approved
>> >> for J-3 is not realistic we also know that we will land significantly
>> more
>> >> features in this milestone than the other two that have already been
>> and
>> >> gone, which to my way of thinking is actually kind of backwards to the
>> ideal
>> >> situation.
>> >>
>> >> What is unclear to me however is how much of this is a result of
>> >> difficulty identifying and approving less controversial/more
>> straightforward
>> >> specifications quickly following summit (keeping in mind this time
>> around
>> >> there was arguably some additional delay as the *-specs repository
>> approach
>> >> was bedded down), an unavoidable result of human nature being to
>> *really*
>> >> push when there is a *hard* deadline to beat, or just that these
>> earlier
>> >> milestones are somewhat impacted from fatigue from the summit (I know
>> a lot
>> >> of people also try to take some well earned time off around this
>> period + of
>> >> course many are still concentrat

Re: [openstack-dev] [Neutron] Specs approved for Juno-3 and exceptions

[Mandeep Dhami]

What would be a good guideline for "timely manner"? I would recommend
something like 2-3 days unless the reviewer is on vacation or is
indisposed. Is it possible to update gerrit/jenkins to send reminders to
reviewers in such a scenario?

Regards,
Mandeep
-




On Fri, Jul 25, 2014 at 3:14 PM, Kyle Mestery  wrote:

> On Fri, Jul 25, 2014 at 4:48 PM, Mandeep Dhami 
> wrote:
> >
> > Thanks for the deck Jay, that is very helpful.
> >
> > Also, would it help the process by having some clear
> guidelines/expectations
> > around review time as well? In particular, if you have put a -1 or -2,
> and
> > the issues that you have identified have been addressed by an update (or
> at
> > least the original author thinks that he has addressed your concern), is
> it
> > reasonable to expect that you will re-review in a "reasonable time"? This
> > way, the updates can either proceed, or be rejected, as they are being
> > developed instead of accumulating in a backlog that we then try to get
> > approved on the last day of the cut-off?
> >
> I agree, if someone puts a -2 on a patch stressing an issue and the
> committer has resolved those issues, the -2 should also be resolved in
> a timely manner. If the issue can't be resolved in the review itself,
> as this wiki page [1] indicates, the issue should be moved to the
> mailing list.
>
> Thanks,
> Kyle
>
> [1] https://wiki.openstack.org/wiki/CodeReviewGuidelines
>
> > Regards,
> > Mandeep
> >
> >
> >
> > On Fri, Jul 25, 2014 at 12:50 PM, Steve Gordon 
> wrote:
> >>
> >> - Original Message -
> >> > From: "Jay Pipes" 
> >> > To: openstack-dev@lists.openstack.org
> >> >
> >> > On 07/24/2014 10:05 AM, CARVER, PAUL wrote:
> >> > > Alan Kavanagh wrote:
> >> > >
> >> > >> If we have more work being put on the table, then more Core
> >> > >> members would definitely go a long way with assisting this, we cant
> >> > >> wait for folks to be reviewing stuff as an excuse to not get
> >> > >> features landed in a given release.
> >> >
> >> > We absolutely can and should wait for folks to be reviewing stuff
> >> > properly. A large number of problems in OpenStack code and flawed
> design
> >> > can be attributed to impatience and pushing through code that wasn't
> >> > ready.
> >> >
> >> > I've said this many times, but the best way to get core reviews on
> >> > patches that you submit is to put the effort into reviewing others'
> >> > code. Core reviewers are more willing to do reviews for someone who is
> >> > clearly trying to help the project in more ways than just pushing
> their
> >> > own code. Note that, Alan, I'm not trying to imply that you are guilty
> >> > of the above! :) I'm just recommending techniques for the general
> >> > contributor community who are not on a core team (including myself!).
> >>
> >> I agree with all of the above, I do think however there is another
> >> un-addressed area where there *may* be room for optimization - which is
> how
> >> we use the earlier milestones. I apologize in advance because this is
> >> somewhat tangential to Alan's points but I think it is relevant to the
> >> general frustration around what did/didn't get approved in time for the
> >> deadline and ultimately what will or wont get reviewed in time to make
> the
> >> release versus being punted to Kilo or even further down the road.
> >>
> >> We land very, very, little in terms of feature work in the *-1 and *-2
> >> milestones in each release (and this is not just a Neutron thing). Even
> >> though we know without a doubt that the amount of work currently
> approved
> >> for J-3 is not realistic we also know that we will land significantly
> more
> >> features in this milestone than the other two that have already been and
> >> gone, which to my way of thinking is actually kind of backwards to the
> ideal
> >> situation.
> >>
> >> What is unclear to me however is how much of this is a result of
> >> difficulty identifying and approving less controversial/more
> straightforward
> >> specifications quickly following summit (keeping in mind this time
> around
> >> there was arguably some additional delay as the *-specs repository
> approach
> >> was bedded down), an unavoidable result of human nature being to
> *really*
> >> push when there is a *hard* deadline to beat, or just that these earlier
> >> milestones are somewhat impacted from fatigue from the summit (I know a
> lot
> >> of people also try to take some well earned time off around this period
> + of
> >> course many are still concentrated on stabilization of the previous
> >> release). As a result it's unclear whether there is anything concrete
> that
> >> can be done to change this but I thought I would bring it up in case
> anyone
> >> else has any bright ideas!
> >>
> >> > [SNIP]
> >>
> >> > > We ought to (in my personal opinion) be supplying core reviewers to
> >> > > at least a couple of OpenStack projects. But one way or another we
> >> > > need to get more capabilities reviewed and merged. My personal 

Re: [openstack-dev] [Neutron] Specs approved for Juno-3 and exceptions

[Kyle Mestery]

On Fri, Jul 25, 2014 at 4:48 PM, Mandeep Dhami  wrote:
>
> Thanks for the deck Jay, that is very helpful.
>
> Also, would it help the process by having some clear guidelines/expectations
> around review time as well? In particular, if you have put a -1 or -2, and
> the issues that you have identified have been addressed by an update (or at
> least the original author thinks that he has addressed your concern), is it
> reasonable to expect that you will re-review in a "reasonable time"? This
> way, the updates can either proceed, or be rejected, as they are being
> developed instead of accumulating in a backlog that we then try to get
> approved on the last day of the cut-off?
>
I agree, if someone puts a -2 on a patch stressing an issue and the
committer has resolved those issues, the -2 should also be resolved in
a timely manner. If the issue can't be resolved in the review itself,
as this wiki page [1] indicates, the issue should be moved to the
mailing list.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/CodeReviewGuidelines

> Regards,
> Mandeep
>
>
>
> On Fri, Jul 25, 2014 at 12:50 PM, Steve Gordon  wrote:
>>
>> - Original Message -
>> > From: "Jay Pipes" 
>> > To: openstack-dev@lists.openstack.org
>> >
>> > On 07/24/2014 10:05 AM, CARVER, PAUL wrote:
>> > > Alan Kavanagh wrote:
>> > >
>> > >> If we have more work being put on the table, then more Core
>> > >> members would definitely go a long way with assisting this, we cant
>> > >> wait for folks to be reviewing stuff as an excuse to not get
>> > >> features landed in a given release.
>> >
>> > We absolutely can and should wait for folks to be reviewing stuff
>> > properly. A large number of problems in OpenStack code and flawed design
>> > can be attributed to impatience and pushing through code that wasn't
>> > ready.
>> >
>> > I've said this many times, but the best way to get core reviews on
>> > patches that you submit is to put the effort into reviewing others'
>> > code. Core reviewers are more willing to do reviews for someone who is
>> > clearly trying to help the project in more ways than just pushing their
>> > own code. Note that, Alan, I'm not trying to imply that you are guilty
>> > of the above! :) I'm just recommending techniques for the general
>> > contributor community who are not on a core team (including myself!).
>>
>> I agree with all of the above, I do think however there is another
>> un-addressed area where there *may* be room for optimization - which is how
>> we use the earlier milestones. I apologize in advance because this is
>> somewhat tangential to Alan's points but I think it is relevant to the
>> general frustration around what did/didn't get approved in time for the
>> deadline and ultimately what will or wont get reviewed in time to make the
>> release versus being punted to Kilo or even further down the road.
>>
>> We land very, very, little in terms of feature work in the *-1 and *-2
>> milestones in each release (and this is not just a Neutron thing). Even
>> though we know without a doubt that the amount of work currently approved
>> for J-3 is not realistic we also know that we will land significantly more
>> features in this milestone than the other two that have already been and
>> gone, which to my way of thinking is actually kind of backwards to the ideal
>> situation.
>>
>> What is unclear to me however is how much of this is a result of
>> difficulty identifying and approving less controversial/more straightforward
>> specifications quickly following summit (keeping in mind this time around
>> there was arguably some additional delay as the *-specs repository approach
>> was bedded down), an unavoidable result of human nature being to *really*
>> push when there is a *hard* deadline to beat, or just that these earlier
>> milestones are somewhat impacted from fatigue from the summit (I know a lot
>> of people also try to take some well earned time off around this period + of
>> course many are still concentrated on stabilization of the previous
>> release). As a result it's unclear whether there is anything concrete that
>> can be done to change this but I thought I would bring it up in case anyone
>> else has any bright ideas!
>>
>> > [SNIP]
>>
>> > > We ought to (in my personal opinion) be supplying core reviewers to
>> > > at least a couple of OpenStack projects. But one way or another we
>> > > need to get more capabilities reviewed and merged. My personal top
>> > > disappointments are with the current state of IPv6, HA, and QoS, but
>> > > I'm sure other folks can list lots of other capabilities that
>> > > they're really going to be frustrated to find lacking in Juno.
>> >
>> > I agree with you. It's not something that is fixable overnight, or by a
>> > small group of people, IMO. It's something that needs to be addressed by
>> > the core project teams, acting as a group in order to reduce review wait
>> > times and ensure that there is responsiveness, transparency and
>> > thoroughn

Re: [openstack-dev] [Neutron] Specs approved for Juno-3 and exceptions

[Mandeep Dhami]

Thanks for the deck Jay, that is very helpful.

Also, would it help the process by having some clear
guidelines/expectations around review time as well? In particular, if you
have put a -1 or -2, and the issues that you have identified have been
addressed by an update (or at least the original author thinks that he has
addressed your concern), is it reasonable to expect that you will re-review
in a "reasonable time"? This way, the updates can either proceed, or be
rejected, as they are being developed instead of accumulating in a backlog
that we then try to get approved on the last day of the cut-off?

Regards,
Mandeep



On Fri, Jul 25, 2014 at 12:50 PM, Steve Gordon  wrote:

> - Original Message -
> > From: "Jay Pipes" 
> > To: openstack-dev@lists.openstack.org
> >
> > On 07/24/2014 10:05 AM, CARVER, PAUL wrote:
> > > Alan Kavanagh wrote:
> > >
> > >> If we have more work being put on the table, then more Core
> > >> members would definitely go a long way with assisting this, we cant
> > >> wait for folks to be reviewing stuff as an excuse to not get
> > >> features landed in a given release.
> >
> > We absolutely can and should wait for folks to be reviewing stuff
> > properly. A large number of problems in OpenStack code and flawed design
> > can be attributed to impatience and pushing through code that wasn't
> ready.
> >
> > I've said this many times, but the best way to get core reviews on
> > patches that you submit is to put the effort into reviewing others'
> > code. Core reviewers are more willing to do reviews for someone who is
> > clearly trying to help the project in more ways than just pushing their
> > own code. Note that, Alan, I'm not trying to imply that you are guilty
> > of the above! :) I'm just recommending techniques for the general
> > contributor community who are not on a core team (including myself!).
>
> I agree with all of the above, I do think however there is another
> un-addressed area where there *may* be room for optimization - which is how
> we use the earlier milestones. I apologize in advance because this is
> somewhat tangential to Alan's points but I think it is relevant to the
> general frustration around what did/didn't get approved in time for the
> deadline and ultimately what will or wont get reviewed in time to make the
> release versus being punted to Kilo or even further down the road.
>
> We land very, very, little in terms of feature work in the *-1 and *-2
> milestones in each release (and this is not just a Neutron thing). Even
> though we know without a doubt that the amount of work currently approved
> for J-3 is not realistic we also know that we will land significantly more
> features in this milestone than the other two that have already been and
> gone, which to my way of thinking is actually kind of backwards to the
> ideal situation.
>
> What is unclear to me however is how much of this is a result of
> difficulty identifying and approving less controversial/more
> straightforward specifications quickly following summit (keeping in mind
> this time around there was arguably some additional delay as the *-specs
> repository approach was bedded down), an unavoidable result of human nature
> being to *really* push when there is a *hard* deadline to beat, or just
> that these earlier milestones are somewhat impacted from fatigue from the
> summit (I know a lot of people also try to take some well earned time off
> around this period + of course many are still concentrated on stabilization
> of the previous release). As a result it's unclear whether there is
> anything concrete that can be done to change this but I thought I would
> bring it up in case anyone else has any bright ideas!
>
> > [SNIP]
>
> > > We ought to (in my personal opinion) be supplying core reviewers to
> > > at least a couple of OpenStack projects. But one way or another we
> > > need to get more capabilities reviewed and merged. My personal top
> > > disappointments are with the current state of IPv6, HA, and QoS, but
> > > I'm sure other folks can list lots of other capabilities that
> > > they're really going to be frustrated to find lacking in Juno.
> >
> > I agree with you. It's not something that is fixable overnight, or by a
> > small group of people, IMO. It's something that needs to be addressed by
> > the core project teams, acting as a group in order to reduce review wait
> > times and ensure that there is responsiveness, transparency and
> > thoroughness to the review (code as well as spec) process.
> >
> > I put together some slides recently that have some insights and
> > (hopefully) some helpful suggestions for both doing and receiving code
> > reviews, as well as staying sane in the era of corporate agendas.
> > Perhaps folks will find it useful:
> >
> > http://bit.ly/navigating-openstack-community
>
> As an aside this is a very well put together deck, thanks for sharing!
>
> -Steve
>
> ___
> OpenStack-dev mailing

Re: [openstack-dev] [Neutron] Specs approved for Juno-3 and exceptions

[Kyle Mestery]

On Fri, Jul 25, 2014 at 2:50 PM, Steve Gordon  wrote:
> - Original Message -
>> From: "Jay Pipes" 
>> To: openstack-dev@lists.openstack.org
>>
>> On 07/24/2014 10:05 AM, CARVER, PAUL wrote:
>> > Alan Kavanagh wrote:
>> >
>> >> If we have more work being put on the table, then more Core
>> >> members would definitely go a long way with assisting this, we cant
>> >> wait for folks to be reviewing stuff as an excuse to not get
>> >> features landed in a given release.
>>
>> We absolutely can and should wait for folks to be reviewing stuff
>> properly. A large number of problems in OpenStack code and flawed design
>> can be attributed to impatience and pushing through code that wasn't ready.
>>
>> I've said this many times, but the best way to get core reviews on
>> patches that you submit is to put the effort into reviewing others'
>> code. Core reviewers are more willing to do reviews for someone who is
>> clearly trying to help the project in more ways than just pushing their
>> own code. Note that, Alan, I'm not trying to imply that you are guilty
>> of the above! :) I'm just recommending techniques for the general
>> contributor community who are not on a core team (including myself!).
>
> I agree with all of the above, I do think however there is another 
> un-addressed area where there *may* be room for optimization - which is how 
> we use the earlier milestones. I apologize in advance because this is 
> somewhat tangential to Alan's points but I think it is relevant to the 
> general frustration around what did/didn't get approved in time for the 
> deadline and ultimately what will or wont get reviewed in time to make the 
> release versus being punted to Kilo or even further down the road.
>
> We land very, very, little in terms of feature work in the *-1 and *-2 
> milestones in each release (and this is not just a Neutron thing). Even 
> though we know without a doubt that the amount of work currently approved for 
> J-3 is not realistic we also know that we will land significantly more 
> features in this milestone than the other two that have already been and 
> gone, which to my way of thinking is actually kind of backwards to the ideal 
> situation.
>
This is how it always is, yes.

> What is unclear to me however is how much of this is a result of difficulty 
> identifying and approving less controversial/more straightforward 
> specifications quickly following summit (keeping in mind this time around 
> there was arguably some additional delay as the *-specs repository approach 
> was bedded down), an unavoidable result of human nature being to *really* 
> push when there is a *hard* deadline to beat, or just that these earlier 
> milestones are somewhat impacted from fatigue from the summit (I know a lot 
> of people also try to take some well earned time off around this period + of 
> course many are still concentrated on stabilization of the previous release). 
> As a result it's unclear whether there is anything concrete that can be done 
> to change this but I thought I would bring it up in case anyone else has any 
> bright ideas!
>
I think it's a little bit of human nature, and a little bit of
stalling. The final milestone for a release is the *final* milestone
for that release. So, a rush is always going to happen. I also think
that I find cores focus on reviews easier near the end. I've tried
experimenting with review assignments near the end of Juno-2 (didn't
work out well), and I'm going to try it again in Juno-3 to see if it
works better there.

The bottom line is that I agree with you, and I'm open to ideas in how
to solve the "final milestone" issue.

Thanks,
Kyle

>> [SNIP]
>
>> > We ought to (in my personal opinion) be supplying core reviewers to
>> > at least a couple of OpenStack projects. But one way or another we
>> > need to get more capabilities reviewed and merged. My personal top
>> > disappointments are with the current state of IPv6, HA, and QoS, but
>> > I'm sure other folks can list lots of other capabilities that
>> > they're really going to be frustrated to find lacking in Juno.
>>
>> I agree with you. It's not something that is fixable overnight, or by a
>> small group of people, IMO. It's something that needs to be addressed by
>> the core project teams, acting as a group in order to reduce review wait
>> times and ensure that there is responsiveness, transparency and
>> thoroughness to the review (code as well as spec) process.
>>
>> I put together some slides recently that have some insights and
>> (hopefully) some helpful suggestions for both doing and receiving code
>> reviews, as well as staying sane in the era of corporate agendas.
>> Perhaps folks will find it useful:
>>
>> http://bit.ly/navigating-openstack-community
>
> As an aside this is a very well put together deck, thanks for sharing!
>
> -Steve
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org

Re: [openstack-dev] [Neutron] Specs approved for Juno-3 and exceptions

[Steve Gordon]

- Original Message -
> From: "Jay Pipes" 
> To: openstack-dev@lists.openstack.org
> 
> On 07/24/2014 10:05 AM, CARVER, PAUL wrote:
> > Alan Kavanagh wrote:
> >
> >> If we have more work being put on the table, then more Core
> >> members would definitely go a long way with assisting this, we cant
> >> wait for folks to be reviewing stuff as an excuse to not get
> >> features landed in a given release.
> 
> We absolutely can and should wait for folks to be reviewing stuff
> properly. A large number of problems in OpenStack code and flawed design
> can be attributed to impatience and pushing through code that wasn't ready.
> 
> I've said this many times, but the best way to get core reviews on
> patches that you submit is to put the effort into reviewing others'
> code. Core reviewers are more willing to do reviews for someone who is
> clearly trying to help the project in more ways than just pushing their
> own code. Note that, Alan, I'm not trying to imply that you are guilty
> of the above! :) I'm just recommending techniques for the general
> contributor community who are not on a core team (including myself!).

I agree with all of the above, I do think however there is another un-addressed 
area where there *may* be room for optimization - which is how we use the 
earlier milestones. I apologize in advance because this is somewhat tangential 
to Alan's points but I think it is relevant to the general frustration around 
what did/didn't get approved in time for the deadline and ultimately what will 
or wont get reviewed in time to make the release versus being punted to Kilo or 
even further down the road.

We land very, very, little in terms of feature work in the *-1 and *-2 
milestones in each release (and this is not just a Neutron thing). Even though 
we know without a doubt that the amount of work currently approved for J-3 is 
not realistic we also know that we will land significantly more features in 
this milestone than the other two that have already been and gone, which to my 
way of thinking is actually kind of backwards to the ideal situation.

What is unclear to me however is how much of this is a result of difficulty 
identifying and approving less controversial/more straightforward 
specifications quickly following summit (keeping in mind this time around there 
was arguably some additional delay as the *-specs repository approach was 
bedded down), an unavoidable result of human nature being to *really* push when 
there is a *hard* deadline to beat, or just that these earlier milestones are 
somewhat impacted from fatigue from the summit (I know a lot of people also try 
to take some well earned time off around this period + of course many are still 
concentrated on stabilization of the previous release). As a result it's 
unclear whether there is anything concrete that can be done to change this but 
I thought I would bring it up in case anyone else has any bright ideas!

> [SNIP]

> > We ought to (in my personal opinion) be supplying core reviewers to
> > at least a couple of OpenStack projects. But one way or another we
> > need to get more capabilities reviewed and merged. My personal top
> > disappointments are with the current state of IPv6, HA, and QoS, but
> > I'm sure other folks can list lots of other capabilities that
> > they're really going to be frustrated to find lacking in Juno.
> 
> I agree with you. It's not something that is fixable overnight, or by a
> small group of people, IMO. It's something that needs to be addressed by
> the core project teams, acting as a group in order to reduce review wait
> times and ensure that there is responsiveness, transparency and
> thoroughness to the review (code as well as spec) process.
> 
> I put together some slides recently that have some insights and
> (hopefully) some helpful suggestions for both doing and receiving code
> reviews, as well as staying sane in the era of corporate agendas.
> Perhaps folks will find it useful:
> 
> http://bit.ly/navigating-openstack-community

As an aside this is a very well put together deck, thanks for sharing!

-Steve

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Specs approved for Juno-3 and exceptions

[Jay Pipes]

On 07/24/2014 10:05 AM, CARVER, PAUL wrote:

Alan Kavanagh wrote:


If we have more work being put on the table, then more Core
members would definitely go a long way with assisting this, we cant
wait for folks to be reviewing stuff as an excuse to not get
features landed in a given release.


We absolutely can and should wait for folks to be reviewing stuff
properly. A large number of problems in OpenStack code and flawed design
can be attributed to impatience and pushing through code that wasn't ready.

I've said this many times, but the best way to get core reviews on
patches that you submit is to put the effort into reviewing others'
code. Core reviewers are more willing to do reviews for someone who is
clearly trying to help the project in more ways than just pushing their 
own code. Note that, Alan, I'm not trying to imply that you are guilty 
of the above! :) I'm just recommending techniques for the general

contributor community who are not on a core team (including myself!).


Stability is absolutely essential so we can't force things through
without adequate review. The automated CI testing in OpenStack is
impressive, but it is far from flawless and even if it worked
perfectly it's still just CI, not AI. There's a large class of
problems that it just can't catch.


Yes, exactly.


I agree with Alan that if there's a discrepancy between the amount
of code that folks would like to land in a release and the number of
core member working hours in a six month period then that is
something the board needs to take an interest in.


Well, technically this is not at all what the OpenStack board is
responsible for. This is likely the purview of the PTLs, the individual
project core teams, and possibly the Technical Committee. The board is
really about company-to-company interests, legal and product marketing
topics, and such.


I think a friendly adversarial approach is healthy for OpenStack.
Specs and code should need to be defended, not just rubber stamped.


++


Having core reviewers critiquing code written by their competitors,
suppliers, or vendors is healthy for the overall code quality.


++


However, simply having specs and code not get reviewed at all due to
a shortage of core reviewers is not healthy and will limit the
success of OpenStack.


Agreed.


I don't really follow Linux kernel development, but a quick search
turned up [1] which seems to indicate at least one additional level
between developer and core (depending on whether we consider Linus
and Andrew levels unto themselves and whether we consider OpenStack
projects as full systems or as subsystems of OpenStack.

Speaking only for myself and not AT&T, I'm disappointed that my
employer doesn't have more developers actively writing code.


As an ex-AT&T-er, I would agree with that sentiment.


We ought to (in my personal opinion) be supplying core reviewers to
at least a couple of OpenStack projects. But one way or another we
need to get more capabilities reviewed and merged. My personal top
disappointments are with the current state of IPv6, HA, and QoS, but
I'm sure other folks can list lots of other capabilities that
they're really going to be frustrated to find lacking in Juno.


I agree with you. It's not something that is fixable overnight, or by a 
small group of people, IMO. It's something that needs to be addressed by 
the core project teams, acting as a group in order to reduce review wait 
times and ensure that there is responsiveness, transparency and 
thoroughness to the review (code as well as spec) process.


I put together some slides recently that have some insights and 
(hopefully) some helpful suggestions for both doing and receiving code 
reviews, as well as staying sane in the era of corporate agendas. 
Perhaps folks will find it useful:


http://bit.ly/navigating-openstack-community

Best,
-jay


[1]
http://techblog.aasisvinayak.com/linux-kernel-development-process-how-it-works/





___ OpenStack-dev
mailing list OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Specs approved for Juno-3 and exceptions

[Stefano Maffulli]

On 07/23/2014 06:22 AM, Kyle Mestery wrote:
> Thanks for sending this out Salvatore. We are way oversubscribed,
> and at this point, I'm in agreement on not letting any new
> exceptions which do not fall under the above guidelines. Given how
> much is already packed in there, this makes the most sense.

The increasing time to merge patches and the increasing backlog was a
topic that came up during the Board meeting on Tuesday. Signals seem to
point at not enough core reviewers in many projects as one of the causes
of these issues.  I have signed up to analyze this more carefully so
that the board can come up with suggestions/requirements to members
organization. Stay tuned for more.

For the short term, though, a careful analysis and exercise in
prioritization together with extra efforts for reviews from the parties
involved would be great.

On 07/24/2014 07:05 AM, CARVER, PAUL wrote:
> I don't really follow Linux kernel development, but a quick search 
> turned up [1] which seems to indicate at least one additional level 
> between

It's hard to drive parallels across such different projects. I would
consider Andrew and Linus our release managers (stable vs current) and
subsystem maintainers the equivalent of our PTLs, the driver maintainers
as our 'core reviewers'. I don't think there are more layers on the
kernel. BTW, I heard that in April OpenStack may have surpassed the
kernel in terms of monthly commits, so we're comparable in size.

> Speaking only for myself and not AT&T, I'm disappointed that my 
> employer doesn't have more developers actively writing code. We ought
> to (in my personal opinion) be supplying core reviewers to at least a
> couple of OpenStack projects.

Yes, I would expect any company the size of ATT be providing at least 1
developer upstream for 10 developers downstream. I'll be looking at some
numbers to check if there is a general behavior around this, mabye come
up with recommendations. Stay tuned.

/stef

-- 
Ask and answer questions on https://ask.openstack.org

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Specs approved for Juno-3 and exceptions

[CARVER, PAUL]

Alan Kavanagh wrote:

>If we have more work being put on the table, then more Core members would
>definitely go a long way with assisting this, we cant wait for folks to be
>reviewing stuff as an excuse to not get features landed in a given release.

Stability is absolutely essential so we can't force things through without
adequate review. The automated CI testing in OpenStack is impressive, but
it is far from flawless and even if it worked perfectly it's still just
CI, not AI. There's a large class of problems that it just can't catch.

I agree with Alan that if there's a discrepancy between the amount of code
that folks would like to land in a release and the number of core member
working hours in a six month period then that is something the board needs
to take an interest in.

I think a friendly adversarial approach is healthy for OpenStack. Specs and
code should need to be defended, not just rubber stamped. Having core
reviewers critiquing code written by their competitors, suppliers, or vendors
is healthy for the overall code quality. However, simply having specs and
code not get reviewed at all due to a shortage of core reviewers is not
healthy and will limit the success of OpenStack.

I don't really follow Linux kernel development, but a quick search turned
up [1] which seems to indicate at least one additional level between
developer and core (depending on whether we consider Linus and Andrew levels
unto themselves and whether we consider OpenStack projects as full systems
or as subsystems of OpenStack.

Speaking only for myself and not AT&T, I'm disappointed that my employer
doesn't have more developers actively writing code. We ought to (in my
personal opinion) be supplying core reviewers to at least a couple of
OpenStack projects. But one way or another we need to get more capabilities
reviewed and merged. My personal top disappointments are with the current
state of IPv6, HA, and QoS, but I'm sure other folks can list lots of other
capabilities that they're really going to be frustrated to find lacking in Juno.

[1] 
http://techblog.aasisvinayak.com/linux-kernel-development-process-how-it-works/



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Specs approved for Juno-3 and exceptions

[Alan Kavanagh]

I find it really hard to comprehend the level of transparency here, or lack 
thereof. It seems to me that when we want to get features into a given release 
we are at the mercy of others and while I do understand that the core team cant 
approve and review everything we also can not wait for another release or 
another release for features that would be of importance for Openstack in 
general. Its very discouraging for other members of the community to have 
certain features which are important for them but maybe not for others being 
demoted and pushed further out.

Also, having a core set of people vote on what is essential for one release to 
the next to me is not very transparent and not very democratic way of working, 
it supports only those who want to guide the community one way. 
While I do see a need for ensuring priority, setting of priority to me is again 
not transparent imho. Also, having folks comment really late on BP's that have 
been progressing and folks working hard to progress them only for at the "last 
minute to get it demoted and then moved to another track" I find this not a 
nice way to work in the community, politics are a way of life but if they are 
going to be used as the rule for Openstack and its releases and a way for 
others to govern within the community I find this really disappointing.

If we have more work being put on the table, then more Core members would 
definitely go a long way with assisting this, we cant wait for folks to be 
reviewing stuff as an excuse to not get features landed in a given release.

I know this will strike a cord with some, but I see too much going on that 
makes me very disappointed so I hope by reaching out others will take note to 
help us improve this process, perhaps this is something the Openstack Board can 
take note of and jump in and try and resolve.

Alan

-Original Message-
From: Kyle Mestery [mailto:mest...@mestery.com] 
Sent: July-23-14 9:23 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Neutron] Specs approved for Juno-3 and exceptions

On Wed, Jul 23, 2014 at 7:28 AM, Salvatore Orlando  wrote:
> I'm sure it is not news to anyone that we already have approved a too 
> many specifications for Juno-3. The PTL made clear indeed that "Low priority"
> blueprints are considered best effort.
>
> However, this already leaves us with 23 medium to high specifications 
> to merge in Juno-3. This is already quite close to what the core team 
> can handle, considering history from previous releases and the fact 
> that there are 3 very big items in the list (new LB APIs, distributed 
> router, and group policies).
>
> I've counted already at least 7 requests for spec freeze exceptions on 
> the mailing list, and it is likely more will come. In order to limit 
> oversubscribing, I would suggest to exclude freeze exceptions requests 
> for items which are not:
> - targeting stability and scalability for Neutron FOSS framework
> - have a "community" interest. By that I do not mean necessarily 
> targeting the FOSS bits, but necessarily have support and interest 
> from a number of teams of neutron contributors.
>
> I don't want to be evil to contributors, but I think it is better to 
> be clear now rather than arriving at the end of Juno-3 and having to 
> tell contributors that unfortunately we were not able to give their 
> patches enough review cycles.
>
Thanks for sending this out Salvatore. We are way oversubscribed, and at this 
point, I'm in agreement on not letting any new exceptions which do not fall 
under the above guidelines. Given how much is already packed in there, this 
makes the most sense.

Thanks,
Kyle

> Salvatore
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Specs approved for Juno-3 and exceptions

[Kyle Mestery]

On Wed, Jul 23, 2014 at 7:28 AM, Salvatore Orlando  wrote:
> I'm sure it is not news to anyone that we already have approved a too many
> specifications for Juno-3. The PTL made clear indeed that "Low priority"
> blueprints are considered best effort.
>
> However, this already leaves us with 23 medium to high specifications to
> merge in Juno-3. This is already quite close to what the core team can
> handle, considering history from previous releases and the fact that there
> are 3 very big items in the list (new LB APIs, distributed router, and group
> policies).
>
> I've counted already at least 7 requests for spec freeze exceptions on the
> mailing list, and it is likely more will come. In order to limit
> oversubscribing, I would suggest to exclude freeze exceptions requests for
> items which are not:
> - targeting stability and scalability for Neutron FOSS framework
> - have a "community" interest. By that I do not mean necessarily targeting
> the FOSS bits, but necessarily have support and interest from a number of
> teams of neutron contributors.
>
> I don't want to be evil to contributors, but I think it is better to be
> clear now rather than arriving at the end of Juno-3 and having to tell
> contributors that unfortunately we were not able to give their patches
> enough review cycles.
>
Thanks for sending this out Salvatore. We are way oversubscribed, and
at this point, I'm in agreement on not letting any new exceptions
which do not fall under the above guidelines. Given how much is
already packed in there, this makes the most sense.

Thanks,
Kyle

> Salvatore
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev