Re: [openstack-dev] [Neutron] Specs approved for Juno-3 and exceptions
On 07/25/2014 05:48 PM, Mandeep Dhami wrote: Thanks for the deck Jay, that is very helpful. Also, would it help the process by having some clear guidelines/expectations around review time as well? In particular, if you have put a -1 or -2, and the issues that you have identified have been addressed by an update (or at least the original author thinks that he has addressed your concern), is it reasonable to expect that you will re-review in a reasonable time? This way, the updates can either proceed, or be rejected, as they are being developed instead of accumulating in a backlog that we then try to get approved on the last day of the cut-off? Guilty as charged, Mandeep. :( If I have failed to re-review something in a timely manner, please don't hesitate to either find me on IRC or send me an email saying hey, don't forget about XYZ. People get behind on reviews and sometimes things slip the mind. A gentle reminder is all that is needed, usually. As for a hard number of days before sending an email notification, that might be possible, but it's not like we all have our vacation reminders linked in to Gerrit ;) I think a more personal email or IRC request for specific reviews is more appropriate. Best, -jay On Fri, Jul 25, 2014 at 12:50 PM, Steve Gordon sgor...@redhat.com mailto:sgor...@redhat.com wrote: - Original Message - From: Jay Pipes jaypi...@gmail.com mailto:jaypi...@gmail.com To: openstack-dev@lists.openstack.org mailto:openstack-dev@lists.openstack.org On 07/24/2014 10:05 AM, CARVER, PAUL wrote: Alan Kavanagh wrote: If we have more work being put on the table, then more Core members would definitely go a long way with assisting this, we cant wait for folks to be reviewing stuff as an excuse to not get features landed in a given release. We absolutely can and should wait for folks to be reviewing stuff properly. A large number of problems in OpenStack code and flawed design can be attributed to impatience and pushing through code that wasn't ready. I've said this many times, but the best way to get core reviews on patches that you submit is to put the effort into reviewing others' code. Core reviewers are more willing to do reviews for someone who is clearly trying to help the project in more ways than just pushing their own code. Note that, Alan, I'm not trying to imply that you are guilty of the above! :) I'm just recommending techniques for the general contributor community who are not on a core team (including myself!). I agree with all of the above, I do think however there is another un-addressed area where there *may* be room for optimization - which is how we use the earlier milestones. I apologize in advance because this is somewhat tangential to Alan's points but I think it is relevant to the general frustration around what did/didn't get approved in time for the deadline and ultimately what will or wont get reviewed in time to make the release versus being punted to Kilo or even further down the road. We land very, very, little in terms of feature work in the *-1 and *-2 milestones in each release (and this is not just a Neutron thing). Even though we know without a doubt that the amount of work currently approved for J-3 is not realistic we also know that we will land significantly more features in this milestone than the other two that have already been and gone, which to my way of thinking is actually kind of backwards to the ideal situation. What is unclear to me however is how much of this is a result of difficulty identifying and approving less controversial/more straightforward specifications quickly following summit (keeping in mind this time around there was arguably some additional delay as the *-specs repository approach was bedded down), an unavoidable result of human nature being to *really* push when there is a *hard* deadline to beat, or just that these earlier milestones are somewhat impacted from fatigue from the summit (I know a lot of people also try to take some well earned time off around this period + of course many are still concentrated on stabilization of the previous release). As a result it's unclear whether there is anything concrete that can be done to change this but I thought I would bring it up in case anyone else has any bright ideas! [SNIP] We ought to (in my personal opinion) be supplying core reviewers to at least a couple of OpenStack projects. But one way or another we need to get more capabilities reviewed and merged. My personal top disappointments are with the current state of IPv6, HA, and QoS, but I'm sure other folks can list lots of other capabilities that they're really going to be
Re: [openstack-dev] [Neutron] Specs approved for Juno-3 and exceptions
Thanks Jay. I agree with your position on it, and that is exactly what I would expect as the process in a collaborative community. That feels like the right way ;-) Unfortunately, there have been situations where we have had to ask a reviewer multiple times to re-review the code (after issues identified in a previous review have been addressed). Then you struggle between am I pestering the reviewer vs. what more can we do/needs to be done, please help us understand - and absence of that feedback leads to discouragement for new contributors and piling up of patches for the big deluge near the cut-off deadlines. My suggestion was to deal with outliers like that. If there was a clear guideline, that facilitated the smooth flow of patches and an automated reminder that did not make the person asking for reviews feel that he/she is pestering, that might help. Or maybe if we update infra to report on avg. number of days that a negative review was not re-reviewed after a new patch, we could just address the outliers when we see them. Just an idea to address the outliers, not the normal flow. Regards, Mandeep - On Sat, Jul 26, 2014 at 10:19 AM, Jay Pipes jaypi...@gmail.com wrote: On 07/25/2014 05:48 PM, Mandeep Dhami wrote: Thanks for the deck Jay, that is very helpful. Also, would it help the process by having some clear guidelines/expectations around review time as well? In particular, if you have put a -1 or -2, and the issues that you have identified have been addressed by an update (or at least the original author thinks that he has addressed your concern), is it reasonable to expect that you will re-review in a reasonable time? This way, the updates can either proceed, or be rejected, as they are being developed instead of accumulating in a backlog that we then try to get approved on the last day of the cut-off? Guilty as charged, Mandeep. :( If I have failed to re-review something in a timely manner, please don't hesitate to either find me on IRC or send me an email saying hey, don't forget about XYZ. People get behind on reviews and sometimes things slip the mind. A gentle reminder is all that is needed, usually. As for a hard number of days before sending an email notification, that might be possible, but it's not like we all have our vacation reminders linked in to Gerrit ;) I think a more personal email or IRC request for specific reviews is more appropriate. Best, -jay On Fri, Jul 25, 2014 at 12:50 PM, Steve Gordon sgor...@redhat.com mailto:sgor...@redhat.com wrote: - Original Message - From: Jay Pipes jaypi...@gmail.com mailto:jaypi...@gmail.com To: openstack-dev@lists.openstack.org mailto:openstack-dev@lists.openstack.org On 07/24/2014 10:05 AM, CARVER, PAUL wrote: Alan Kavanagh wrote: If we have more work being put on the table, then more Core members would definitely go a long way with assisting this, we cant wait for folks to be reviewing stuff as an excuse to not get features landed in a given release. We absolutely can and should wait for folks to be reviewing stuff properly. A large number of problems in OpenStack code and flawed design can be attributed to impatience and pushing through code that wasn't ready. I've said this many times, but the best way to get core reviews on patches that you submit is to put the effort into reviewing others' code. Core reviewers are more willing to do reviews for someone who is clearly trying to help the project in more ways than just pushing their own code. Note that, Alan, I'm not trying to imply that you are guilty of the above! :) I'm just recommending techniques for the general contributor community who are not on a core team (including myself!). I agree with all of the above, I do think however there is another un-addressed area where there *may* be room for optimization - which is how we use the earlier milestones. I apologize in advance because this is somewhat tangential to Alan's points but I think it is relevant to the general frustration around what did/didn't get approved in time for the deadline and ultimately what will or wont get reviewed in time to make the release versus being punted to Kilo or even further down the road. We land very, very, little in terms of feature work in the *-1 and *-2 milestones in each release (and this is not just a Neutron thing). Even though we know without a doubt that the amount of work currently approved for J-3 is not realistic we also know that we will land significantly more features in this milestone than the other two that have already been and gone, which to my way of thinking is actually kind of backwards to the ideal situation. What is unclear to me however is how much of
Re: [openstack-dev] [Neutron] Specs approved for Juno-3 and exceptions
- Original Message - From: Jay Pipes jaypi...@gmail.com To: openstack-dev@lists.openstack.org On 07/24/2014 10:05 AM, CARVER, PAUL wrote: Alan Kavanagh wrote: If we have more work being put on the table, then more Core members would definitely go a long way with assisting this, we cant wait for folks to be reviewing stuff as an excuse to not get features landed in a given release. We absolutely can and should wait for folks to be reviewing stuff properly. A large number of problems in OpenStack code and flawed design can be attributed to impatience and pushing through code that wasn't ready. I've said this many times, but the best way to get core reviews on patches that you submit is to put the effort into reviewing others' code. Core reviewers are more willing to do reviews for someone who is clearly trying to help the project in more ways than just pushing their own code. Note that, Alan, I'm not trying to imply that you are guilty of the above! :) I'm just recommending techniques for the general contributor community who are not on a core team (including myself!). I agree with all of the above, I do think however there is another un-addressed area where there *may* be room for optimization - which is how we use the earlier milestones. I apologize in advance because this is somewhat tangential to Alan's points but I think it is relevant to the general frustration around what did/didn't get approved in time for the deadline and ultimately what will or wont get reviewed in time to make the release versus being punted to Kilo or even further down the road. We land very, very, little in terms of feature work in the *-1 and *-2 milestones in each release (and this is not just a Neutron thing). Even though we know without a doubt that the amount of work currently approved for J-3 is not realistic we also know that we will land significantly more features in this milestone than the other two that have already been and gone, which to my way of thinking is actually kind of backwards to the ideal situation. What is unclear to me however is how much of this is a result of difficulty identifying and approving less controversial/more straightforward specifications quickly following summit (keeping in mind this time around there was arguably some additional delay as the *-specs repository approach was bedded down), an unavoidable result of human nature being to *really* push when there is a *hard* deadline to beat, or just that these earlier milestones are somewhat impacted from fatigue from the summit (I know a lot of people also try to take some well earned time off around this period + of course many are still concentrated on stabilization of the previous release). As a result it's unclear whether there is anything concrete that can be done to change this but I thought I would bring it up in case anyone else has any bright ideas! [SNIP] We ought to (in my personal opinion) be supplying core reviewers to at least a couple of OpenStack projects. But one way or another we need to get more capabilities reviewed and merged. My personal top disappointments are with the current state of IPv6, HA, and QoS, but I'm sure other folks can list lots of other capabilities that they're really going to be frustrated to find lacking in Juno. I agree with you. It's not something that is fixable overnight, or by a small group of people, IMO. It's something that needs to be addressed by the core project teams, acting as a group in order to reduce review wait times and ensure that there is responsiveness, transparency and thoroughness to the review (code as well as spec) process. I put together some slides recently that have some insights and (hopefully) some helpful suggestions for both doing and receiving code reviews, as well as staying sane in the era of corporate agendas. Perhaps folks will find it useful: http://bit.ly/navigating-openstack-community As an aside this is a very well put together deck, thanks for sharing! -Steve ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] Specs approved for Juno-3 and exceptions
On Fri, Jul 25, 2014 at 2:50 PM, Steve Gordon sgor...@redhat.com wrote: - Original Message - From: Jay Pipes jaypi...@gmail.com To: openstack-dev@lists.openstack.org On 07/24/2014 10:05 AM, CARVER, PAUL wrote: Alan Kavanagh wrote: If we have more work being put on the table, then more Core members would definitely go a long way with assisting this, we cant wait for folks to be reviewing stuff as an excuse to not get features landed in a given release. We absolutely can and should wait for folks to be reviewing stuff properly. A large number of problems in OpenStack code and flawed design can be attributed to impatience and pushing through code that wasn't ready. I've said this many times, but the best way to get core reviews on patches that you submit is to put the effort into reviewing others' code. Core reviewers are more willing to do reviews for someone who is clearly trying to help the project in more ways than just pushing their own code. Note that, Alan, I'm not trying to imply that you are guilty of the above! :) I'm just recommending techniques for the general contributor community who are not on a core team (including myself!). I agree with all of the above, I do think however there is another un-addressed area where there *may* be room for optimization - which is how we use the earlier milestones. I apologize in advance because this is somewhat tangential to Alan's points but I think it is relevant to the general frustration around what did/didn't get approved in time for the deadline and ultimately what will or wont get reviewed in time to make the release versus being punted to Kilo or even further down the road. We land very, very, little in terms of feature work in the *-1 and *-2 milestones in each release (and this is not just a Neutron thing). Even though we know without a doubt that the amount of work currently approved for J-3 is not realistic we also know that we will land significantly more features in this milestone than the other two that have already been and gone, which to my way of thinking is actually kind of backwards to the ideal situation. This is how it always is, yes. What is unclear to me however is how much of this is a result of difficulty identifying and approving less controversial/more straightforward specifications quickly following summit (keeping in mind this time around there was arguably some additional delay as the *-specs repository approach was bedded down), an unavoidable result of human nature being to *really* push when there is a *hard* deadline to beat, or just that these earlier milestones are somewhat impacted from fatigue from the summit (I know a lot of people also try to take some well earned time off around this period + of course many are still concentrated on stabilization of the previous release). As a result it's unclear whether there is anything concrete that can be done to change this but I thought I would bring it up in case anyone else has any bright ideas! I think it's a little bit of human nature, and a little bit of stalling. The final milestone for a release is the *final* milestone for that release. So, a rush is always going to happen. I also think that I find cores focus on reviews easier near the end. I've tried experimenting with review assignments near the end of Juno-2 (didn't work out well), and I'm going to try it again in Juno-3 to see if it works better there. The bottom line is that I agree with you, and I'm open to ideas in how to solve the final milestone issue. Thanks, Kyle [SNIP] We ought to (in my personal opinion) be supplying core reviewers to at least a couple of OpenStack projects. But one way or another we need to get more capabilities reviewed and merged. My personal top disappointments are with the current state of IPv6, HA, and QoS, but I'm sure other folks can list lots of other capabilities that they're really going to be frustrated to find lacking in Juno. I agree with you. It's not something that is fixable overnight, or by a small group of people, IMO. It's something that needs to be addressed by the core project teams, acting as a group in order to reduce review wait times and ensure that there is responsiveness, transparency and thoroughness to the review (code as well as spec) process. I put together some slides recently that have some insights and (hopefully) some helpful suggestions for both doing and receiving code reviews, as well as staying sane in the era of corporate agendas. Perhaps folks will find it useful: http://bit.ly/navigating-openstack-community As an aside this is a very well put together deck, thanks for sharing! -Steve ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev
Re: [openstack-dev] [Neutron] Specs approved for Juno-3 and exceptions
Thanks for the deck Jay, that is very helpful. Also, would it help the process by having some clear guidelines/expectations around review time as well? In particular, if you have put a -1 or -2, and the issues that you have identified have been addressed by an update (or at least the original author thinks that he has addressed your concern), is it reasonable to expect that you will re-review in a reasonable time? This way, the updates can either proceed, or be rejected, as they are being developed instead of accumulating in a backlog that we then try to get approved on the last day of the cut-off? Regards, Mandeep On Fri, Jul 25, 2014 at 12:50 PM, Steve Gordon sgor...@redhat.com wrote: - Original Message - From: Jay Pipes jaypi...@gmail.com To: openstack-dev@lists.openstack.org On 07/24/2014 10:05 AM, CARVER, PAUL wrote: Alan Kavanagh wrote: If we have more work being put on the table, then more Core members would definitely go a long way with assisting this, we cant wait for folks to be reviewing stuff as an excuse to not get features landed in a given release. We absolutely can and should wait for folks to be reviewing stuff properly. A large number of problems in OpenStack code and flawed design can be attributed to impatience and pushing through code that wasn't ready. I've said this many times, but the best way to get core reviews on patches that you submit is to put the effort into reviewing others' code. Core reviewers are more willing to do reviews for someone who is clearly trying to help the project in more ways than just pushing their own code. Note that, Alan, I'm not trying to imply that you are guilty of the above! :) I'm just recommending techniques for the general contributor community who are not on a core team (including myself!). I agree with all of the above, I do think however there is another un-addressed area where there *may* be room for optimization - which is how we use the earlier milestones. I apologize in advance because this is somewhat tangential to Alan's points but I think it is relevant to the general frustration around what did/didn't get approved in time for the deadline and ultimately what will or wont get reviewed in time to make the release versus being punted to Kilo or even further down the road. We land very, very, little in terms of feature work in the *-1 and *-2 milestones in each release (and this is not just a Neutron thing). Even though we know without a doubt that the amount of work currently approved for J-3 is not realistic we also know that we will land significantly more features in this milestone than the other two that have already been and gone, which to my way of thinking is actually kind of backwards to the ideal situation. What is unclear to me however is how much of this is a result of difficulty identifying and approving less controversial/more straightforward specifications quickly following summit (keeping in mind this time around there was arguably some additional delay as the *-specs repository approach was bedded down), an unavoidable result of human nature being to *really* push when there is a *hard* deadline to beat, or just that these earlier milestones are somewhat impacted from fatigue from the summit (I know a lot of people also try to take some well earned time off around this period + of course many are still concentrated on stabilization of the previous release). As a result it's unclear whether there is anything concrete that can be done to change this but I thought I would bring it up in case anyone else has any bright ideas! [SNIP] We ought to (in my personal opinion) be supplying core reviewers to at least a couple of OpenStack projects. But one way or another we need to get more capabilities reviewed and merged. My personal top disappointments are with the current state of IPv6, HA, and QoS, but I'm sure other folks can list lots of other capabilities that they're really going to be frustrated to find lacking in Juno. I agree with you. It's not something that is fixable overnight, or by a small group of people, IMO. It's something that needs to be addressed by the core project teams, acting as a group in order to reduce review wait times and ensure that there is responsiveness, transparency and thoroughness to the review (code as well as spec) process. I put together some slides recently that have some insights and (hopefully) some helpful suggestions for both doing and receiving code reviews, as well as staying sane in the era of corporate agendas. Perhaps folks will find it useful: http://bit.ly/navigating-openstack-community As an aside this is a very well put together deck, thanks for sharing! -Steve ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] Specs approved for Juno-3 and exceptions
On Fri, Jul 25, 2014 at 4:48 PM, Mandeep Dhami dh...@noironetworks.com wrote: Thanks for the deck Jay, that is very helpful. Also, would it help the process by having some clear guidelines/expectations around review time as well? In particular, if you have put a -1 or -2, and the issues that you have identified have been addressed by an update (or at least the original author thinks that he has addressed your concern), is it reasonable to expect that you will re-review in a reasonable time? This way, the updates can either proceed, or be rejected, as they are being developed instead of accumulating in a backlog that we then try to get approved on the last day of the cut-off? I agree, if someone puts a -2 on a patch stressing an issue and the committer has resolved those issues, the -2 should also be resolved in a timely manner. If the issue can't be resolved in the review itself, as this wiki page [1] indicates, the issue should be moved to the mailing list. Thanks, Kyle [1] https://wiki.openstack.org/wiki/CodeReviewGuidelines Regards, Mandeep On Fri, Jul 25, 2014 at 12:50 PM, Steve Gordon sgor...@redhat.com wrote: - Original Message - From: Jay Pipes jaypi...@gmail.com To: openstack-dev@lists.openstack.org On 07/24/2014 10:05 AM, CARVER, PAUL wrote: Alan Kavanagh wrote: If we have more work being put on the table, then more Core members would definitely go a long way with assisting this, we cant wait for folks to be reviewing stuff as an excuse to not get features landed in a given release. We absolutely can and should wait for folks to be reviewing stuff properly. A large number of problems in OpenStack code and flawed design can be attributed to impatience and pushing through code that wasn't ready. I've said this many times, but the best way to get core reviews on patches that you submit is to put the effort into reviewing others' code. Core reviewers are more willing to do reviews for someone who is clearly trying to help the project in more ways than just pushing their own code. Note that, Alan, I'm not trying to imply that you are guilty of the above! :) I'm just recommending techniques for the general contributor community who are not on a core team (including myself!). I agree with all of the above, I do think however there is another un-addressed area where there *may* be room for optimization - which is how we use the earlier milestones. I apologize in advance because this is somewhat tangential to Alan's points but I think it is relevant to the general frustration around what did/didn't get approved in time for the deadline and ultimately what will or wont get reviewed in time to make the release versus being punted to Kilo or even further down the road. We land very, very, little in terms of feature work in the *-1 and *-2 milestones in each release (and this is not just a Neutron thing). Even though we know without a doubt that the amount of work currently approved for J-3 is not realistic we also know that we will land significantly more features in this milestone than the other two that have already been and gone, which to my way of thinking is actually kind of backwards to the ideal situation. What is unclear to me however is how much of this is a result of difficulty identifying and approving less controversial/more straightforward specifications quickly following summit (keeping in mind this time around there was arguably some additional delay as the *-specs repository approach was bedded down), an unavoidable result of human nature being to *really* push when there is a *hard* deadline to beat, or just that these earlier milestones are somewhat impacted from fatigue from the summit (I know a lot of people also try to take some well earned time off around this period + of course many are still concentrated on stabilization of the previous release). As a result it's unclear whether there is anything concrete that can be done to change this but I thought I would bring it up in case anyone else has any bright ideas! [SNIP] We ought to (in my personal opinion) be supplying core reviewers to at least a couple of OpenStack projects. But one way or another we need to get more capabilities reviewed and merged. My personal top disappointments are with the current state of IPv6, HA, and QoS, but I'm sure other folks can list lots of other capabilities that they're really going to be frustrated to find lacking in Juno. I agree with you. It's not something that is fixable overnight, or by a small group of people, IMO. It's something that needs to be addressed by the core project teams, acting as a group in order to reduce review wait times and ensure that there is responsiveness, transparency and thoroughness to the review (code as well as spec) process. I put together some slides recently that have some insights and (hopefully) some helpful
Re: [openstack-dev] [Neutron] Specs approved for Juno-3 and exceptions
What would be a good guideline for timely manner? I would recommend something like 2-3 days unless the reviewer is on vacation or is indisposed. Is it possible to update gerrit/jenkins to send reminders to reviewers in such a scenario? Regards, Mandeep - On Fri, Jul 25, 2014 at 3:14 PM, Kyle Mestery mest...@mestery.com wrote: On Fri, Jul 25, 2014 at 4:48 PM, Mandeep Dhami dh...@noironetworks.com wrote: Thanks for the deck Jay, that is very helpful. Also, would it help the process by having some clear guidelines/expectations around review time as well? In particular, if you have put a -1 or -2, and the issues that you have identified have been addressed by an update (or at least the original author thinks that he has addressed your concern), is it reasonable to expect that you will re-review in a reasonable time? This way, the updates can either proceed, or be rejected, as they are being developed instead of accumulating in a backlog that we then try to get approved on the last day of the cut-off? I agree, if someone puts a -2 on a patch stressing an issue and the committer has resolved those issues, the -2 should also be resolved in a timely manner. If the issue can't be resolved in the review itself, as this wiki page [1] indicates, the issue should be moved to the mailing list. Thanks, Kyle [1] https://wiki.openstack.org/wiki/CodeReviewGuidelines Regards, Mandeep On Fri, Jul 25, 2014 at 12:50 PM, Steve Gordon sgor...@redhat.com wrote: - Original Message - From: Jay Pipes jaypi...@gmail.com To: openstack-dev@lists.openstack.org On 07/24/2014 10:05 AM, CARVER, PAUL wrote: Alan Kavanagh wrote: If we have more work being put on the table, then more Core members would definitely go a long way with assisting this, we cant wait for folks to be reviewing stuff as an excuse to not get features landed in a given release. We absolutely can and should wait for folks to be reviewing stuff properly. A large number of problems in OpenStack code and flawed design can be attributed to impatience and pushing through code that wasn't ready. I've said this many times, but the best way to get core reviews on patches that you submit is to put the effort into reviewing others' code. Core reviewers are more willing to do reviews for someone who is clearly trying to help the project in more ways than just pushing their own code. Note that, Alan, I'm not trying to imply that you are guilty of the above! :) I'm just recommending techniques for the general contributor community who are not on a core team (including myself!). I agree with all of the above, I do think however there is another un-addressed area where there *may* be room for optimization - which is how we use the earlier milestones. I apologize in advance because this is somewhat tangential to Alan's points but I think it is relevant to the general frustration around what did/didn't get approved in time for the deadline and ultimately what will or wont get reviewed in time to make the release versus being punted to Kilo or even further down the road. We land very, very, little in terms of feature work in the *-1 and *-2 milestones in each release (and this is not just a Neutron thing). Even though we know without a doubt that the amount of work currently approved for J-3 is not realistic we also know that we will land significantly more features in this milestone than the other two that have already been and gone, which to my way of thinking is actually kind of backwards to the ideal situation. What is unclear to me however is how much of this is a result of difficulty identifying and approving less controversial/more straightforward specifications quickly following summit (keeping in mind this time around there was arguably some additional delay as the *-specs repository approach was bedded down), an unavoidable result of human nature being to *really* push when there is a *hard* deadline to beat, or just that these earlier milestones are somewhat impacted from fatigue from the summit (I know a lot of people also try to take some well earned time off around this period + of course many are still concentrated on stabilization of the previous release). As a result it's unclear whether there is anything concrete that can be done to change this but I thought I would bring it up in case anyone else has any bright ideas! [SNIP] We ought to (in my personal opinion) be supplying core reviewers to at least a couple of OpenStack projects. But one way or another we need to get more capabilities reviewed and merged. My personal top disappointments are with the current state of IPv6, HA, and QoS, but I'm sure other folks can list lots of other capabilities that they're really going to be frustrated to find lacking in Juno.
Re: [openstack-dev] [Neutron] Specs approved for Juno-3 and exceptions
I agree that it's important to set a guideline for this topic. What if the said reviewer is on vacation or indisposed? Should a fallback strategy exist for that case? A reviewer could indicate a delegate core to review its -2s whenever he has no chance to do it. Thanks, Ivar. On Fri, Jul 25, 2014 at 5:35 PM, Mandeep Dhami dh...@noironetworks.com wrote: What would be a good guideline for timely manner? I would recommend something like 2-3 days unless the reviewer is on vacation or is indisposed. Is it possible to update gerrit/jenkins to send reminders to reviewers in such a scenario? Regards, Mandeep - On Fri, Jul 25, 2014 at 3:14 PM, Kyle Mestery mest...@mestery.com wrote: On Fri, Jul 25, 2014 at 4:48 PM, Mandeep Dhami dh...@noironetworks.com wrote: Thanks for the deck Jay, that is very helpful. Also, would it help the process by having some clear guidelines/expectations around review time as well? In particular, if you have put a -1 or -2, and the issues that you have identified have been addressed by an update (or at least the original author thinks that he has addressed your concern), is it reasonable to expect that you will re-review in a reasonable time? This way, the updates can either proceed, or be rejected, as they are being developed instead of accumulating in a backlog that we then try to get approved on the last day of the cut-off? I agree, if someone puts a -2 on a patch stressing an issue and the committer has resolved those issues, the -2 should also be resolved in a timely manner. If the issue can't be resolved in the review itself, as this wiki page [1] indicates, the issue should be moved to the mailing list. Thanks, Kyle [1] https://wiki.openstack.org/wiki/CodeReviewGuidelines Regards, Mandeep On Fri, Jul 25, 2014 at 12:50 PM, Steve Gordon sgor...@redhat.com wrote: - Original Message - From: Jay Pipes jaypi...@gmail.com To: openstack-dev@lists.openstack.org On 07/24/2014 10:05 AM, CARVER, PAUL wrote: Alan Kavanagh wrote: If we have more work being put on the table, then more Core members would definitely go a long way with assisting this, we cant wait for folks to be reviewing stuff as an excuse to not get features landed in a given release. We absolutely can and should wait for folks to be reviewing stuff properly. A large number of problems in OpenStack code and flawed design can be attributed to impatience and pushing through code that wasn't ready. I've said this many times, but the best way to get core reviews on patches that you submit is to put the effort into reviewing others' code. Core reviewers are more willing to do reviews for someone who is clearly trying to help the project in more ways than just pushing their own code. Note that, Alan, I'm not trying to imply that you are guilty of the above! :) I'm just recommending techniques for the general contributor community who are not on a core team (including myself!). I agree with all of the above, I do think however there is another un-addressed area where there *may* be room for optimization - which is how we use the earlier milestones. I apologize in advance because this is somewhat tangential to Alan's points but I think it is relevant to the general frustration around what did/didn't get approved in time for the deadline and ultimately what will or wont get reviewed in time to make the release versus being punted to Kilo or even further down the road. We land very, very, little in terms of feature work in the *-1 and *-2 milestones in each release (and this is not just a Neutron thing). Even though we know without a doubt that the amount of work currently approved for J-3 is not realistic we also know that we will land significantly more features in this milestone than the other two that have already been and gone, which to my way of thinking is actually kind of backwards to the ideal situation. What is unclear to me however is how much of this is a result of difficulty identifying and approving less controversial/more straightforward specifications quickly following summit (keeping in mind this time around there was arguably some additional delay as the *-specs repository approach was bedded down), an unavoidable result of human nature being to *really* push when there is a *hard* deadline to beat, or just that these earlier milestones are somewhat impacted from fatigue from the summit (I know a lot of people also try to take some well earned time off around this period + of course many are still concentrated on stabilization of the previous release). As a result it's unclear whether there is anything concrete that can be done to change this but I thought I would bring it up in case anyone else has any bright ideas! [SNIP] We ought to (in my personal opinion) be
Re: [openstack-dev] [Neutron] Specs approved for Juno-3 and exceptions
Alan Kavanagh wrote: If we have more work being put on the table, then more Core members would definitely go a long way with assisting this, we cant wait for folks to be reviewing stuff as an excuse to not get features landed in a given release. Stability is absolutely essential so we can't force things through without adequate review. The automated CI testing in OpenStack is impressive, but it is far from flawless and even if it worked perfectly it's still just CI, not AI. There's a large class of problems that it just can't catch. I agree with Alan that if there's a discrepancy between the amount of code that folks would like to land in a release and the number of core member working hours in a six month period then that is something the board needs to take an interest in. I think a friendly adversarial approach is healthy for OpenStack. Specs and code should need to be defended, not just rubber stamped. Having core reviewers critiquing code written by their competitors, suppliers, or vendors is healthy for the overall code quality. However, simply having specs and code not get reviewed at all due to a shortage of core reviewers is not healthy and will limit the success of OpenStack. I don't really follow Linux kernel development, but a quick search turned up [1] which seems to indicate at least one additional level between developer and core (depending on whether we consider Linus and Andrew levels unto themselves and whether we consider OpenStack projects as full systems or as subsystems of OpenStack. Speaking only for myself and not ATT, I'm disappointed that my employer doesn't have more developers actively writing code. We ought to (in my personal opinion) be supplying core reviewers to at least a couple of OpenStack projects. But one way or another we need to get more capabilities reviewed and merged. My personal top disappointments are with the current state of IPv6, HA, and QoS, but I'm sure other folks can list lots of other capabilities that they're really going to be frustrated to find lacking in Juno. [1] http://techblog.aasisvinayak.com/linux-kernel-development-process-how-it-works/ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] Specs approved for Juno-3 and exceptions
On 07/23/2014 06:22 AM, Kyle Mestery wrote: Thanks for sending this out Salvatore. We are way oversubscribed, and at this point, I'm in agreement on not letting any new exceptions which do not fall under the above guidelines. Given how much is already packed in there, this makes the most sense. The increasing time to merge patches and the increasing backlog was a topic that came up during the Board meeting on Tuesday. Signals seem to point at not enough core reviewers in many projects as one of the causes of these issues. I have signed up to analyze this more carefully so that the board can come up with suggestions/requirements to members organization. Stay tuned for more. For the short term, though, a careful analysis and exercise in prioritization together with extra efforts for reviews from the parties involved would be great. On 07/24/2014 07:05 AM, CARVER, PAUL wrote: I don't really follow Linux kernel development, but a quick search turned up [1] which seems to indicate at least one additional level between It's hard to drive parallels across such different projects. I would consider Andrew and Linus our release managers (stable vs current) and subsystem maintainers the equivalent of our PTLs, the driver maintainers as our 'core reviewers'. I don't think there are more layers on the kernel. BTW, I heard that in April OpenStack may have surpassed the kernel in terms of monthly commits, so we're comparable in size. Speaking only for myself and not ATT, I'm disappointed that my employer doesn't have more developers actively writing code. We ought to (in my personal opinion) be supplying core reviewers to at least a couple of OpenStack projects. Yes, I would expect any company the size of ATT be providing at least 1 developer upstream for 10 developers downstream. I'll be looking at some numbers to check if there is a general behavior around this, mabye come up with recommendations. Stay tuned. /stef -- Ask and answer questions on https://ask.openstack.org ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] Specs approved for Juno-3 and exceptions
On 07/24/2014 10:05 AM, CARVER, PAUL wrote: Alan Kavanagh wrote: If we have more work being put on the table, then more Core members would definitely go a long way with assisting this, we cant wait for folks to be reviewing stuff as an excuse to not get features landed in a given release. We absolutely can and should wait for folks to be reviewing stuff properly. A large number of problems in OpenStack code and flawed design can be attributed to impatience and pushing through code that wasn't ready. I've said this many times, but the best way to get core reviews on patches that you submit is to put the effort into reviewing others' code. Core reviewers are more willing to do reviews for someone who is clearly trying to help the project in more ways than just pushing their own code. Note that, Alan, I'm not trying to imply that you are guilty of the above! :) I'm just recommending techniques for the general contributor community who are not on a core team (including myself!). Stability is absolutely essential so we can't force things through without adequate review. The automated CI testing in OpenStack is impressive, but it is far from flawless and even if it worked perfectly it's still just CI, not AI. There's a large class of problems that it just can't catch. Yes, exactly. I agree with Alan that if there's a discrepancy between the amount of code that folks would like to land in a release and the number of core member working hours in a six month period then that is something the board needs to take an interest in. Well, technically this is not at all what the OpenStack board is responsible for. This is likely the purview of the PTLs, the individual project core teams, and possibly the Technical Committee. The board is really about company-to-company interests, legal and product marketing topics, and such. I think a friendly adversarial approach is healthy for OpenStack. Specs and code should need to be defended, not just rubber stamped. ++ Having core reviewers critiquing code written by their competitors, suppliers, or vendors is healthy for the overall code quality. ++ However, simply having specs and code not get reviewed at all due to a shortage of core reviewers is not healthy and will limit the success of OpenStack. Agreed. I don't really follow Linux kernel development, but a quick search turned up [1] which seems to indicate at least one additional level between developer and core (depending on whether we consider Linus and Andrew levels unto themselves and whether we consider OpenStack projects as full systems or as subsystems of OpenStack. Speaking only for myself and not ATT, I'm disappointed that my employer doesn't have more developers actively writing code. As an ex-ATT-er, I would agree with that sentiment. We ought to (in my personal opinion) be supplying core reviewers to at least a couple of OpenStack projects. But one way or another we need to get more capabilities reviewed and merged. My personal top disappointments are with the current state of IPv6, HA, and QoS, but I'm sure other folks can list lots of other capabilities that they're really going to be frustrated to find lacking in Juno. I agree with you. It's not something that is fixable overnight, or by a small group of people, IMO. It's something that needs to be addressed by the core project teams, acting as a group in order to reduce review wait times and ensure that there is responsiveness, transparency and thoroughness to the review (code as well as spec) process. I put together some slides recently that have some insights and (hopefully) some helpful suggestions for both doing and receiving code reviews, as well as staying sane in the era of corporate agendas. Perhaps folks will find it useful: http://bit.ly/navigating-openstack-community Best, -jay [1] http://techblog.aasisvinayak.com/linux-kernel-development-process-how-it-works/ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] Specs approved for Juno-3 and exceptions
On Wed, Jul 23, 2014 at 7:28 AM, Salvatore Orlando sorla...@nicira.com wrote: I'm sure it is not news to anyone that we already have approved a too many specifications for Juno-3. The PTL made clear indeed that Low priority blueprints are considered best effort. However, this already leaves us with 23 medium to high specifications to merge in Juno-3. This is already quite close to what the core team can handle, considering history from previous releases and the fact that there are 3 very big items in the list (new LB APIs, distributed router, and group policies). I've counted already at least 7 requests for spec freeze exceptions on the mailing list, and it is likely more will come. In order to limit oversubscribing, I would suggest to exclude freeze exceptions requests for items which are not: - targeting stability and scalability for Neutron FOSS framework - have a community interest. By that I do not mean necessarily targeting the FOSS bits, but necessarily have support and interest from a number of teams of neutron contributors. I don't want to be evil to contributors, but I think it is better to be clear now rather than arriving at the end of Juno-3 and having to tell contributors that unfortunately we were not able to give their patches enough review cycles. Thanks for sending this out Salvatore. We are way oversubscribed, and at this point, I'm in agreement on not letting any new exceptions which do not fall under the above guidelines. Given how much is already packed in there, this makes the most sense. Thanks, Kyle Salvatore ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] Specs approved for Juno-3 and exceptions
I find it really hard to comprehend the level of transparency here, or lack thereof. It seems to me that when we want to get features into a given release we are at the mercy of others and while I do understand that the core team cant approve and review everything we also can not wait for another release or another release for features that would be of importance for Openstack in general. Its very discouraging for other members of the community to have certain features which are important for them but maybe not for others being demoted and pushed further out. Also, having a core set of people vote on what is essential for one release to the next to me is not very transparent and not very democratic way of working, it supports only those who want to guide the community one way. While I do see a need for ensuring priority, setting of priority to me is again not transparent imho. Also, having folks comment really late on BP's that have been progressing and folks working hard to progress them only for at the last minute to get it demoted and then moved to another track I find this not a nice way to work in the community, politics are a way of life but if they are going to be used as the rule for Openstack and its releases and a way for others to govern within the community I find this really disappointing. If we have more work being put on the table, then more Core members would definitely go a long way with assisting this, we cant wait for folks to be reviewing stuff as an excuse to not get features landed in a given release. I know this will strike a cord with some, but I see too much going on that makes me very disappointed so I hope by reaching out others will take note to help us improve this process, perhaps this is something the Openstack Board can take note of and jump in and try and resolve. Alan -Original Message- From: Kyle Mestery [mailto:mest...@mestery.com] Sent: July-23-14 9:23 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Neutron] Specs approved for Juno-3 and exceptions On Wed, Jul 23, 2014 at 7:28 AM, Salvatore Orlando sorla...@nicira.com wrote: I'm sure it is not news to anyone that we already have approved a too many specifications for Juno-3. The PTL made clear indeed that Low priority blueprints are considered best effort. However, this already leaves us with 23 medium to high specifications to merge in Juno-3. This is already quite close to what the core team can handle, considering history from previous releases and the fact that there are 3 very big items in the list (new LB APIs, distributed router, and group policies). I've counted already at least 7 requests for spec freeze exceptions on the mailing list, and it is likely more will come. In order to limit oversubscribing, I would suggest to exclude freeze exceptions requests for items which are not: - targeting stability and scalability for Neutron FOSS framework - have a community interest. By that I do not mean necessarily targeting the FOSS bits, but necessarily have support and interest from a number of teams of neutron contributors. I don't want to be evil to contributors, but I think it is better to be clear now rather than arriving at the end of Juno-3 and having to tell contributors that unfortunately we were not able to give their patches enough review cycles. Thanks for sending this out Salvatore. We are way oversubscribed, and at this point, I'm in agreement on not letting any new exceptions which do not fall under the above guidelines. Given how much is already packed in there, this makes the most sense. Thanks, Kyle Salvatore ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
