Re: [openstack-dev] [nova] review runways check-in and feedback
On Wed, Jun 13, 2018 at 10:33 PM, melanie witt wrote: Howdy everyone, We've been experimenting with a new process this cycle, Review Runways [1] and we're about at the middle of the cycle now as we had the r-2 milestone last week June 7. I wanted to start a thread and gather thoughts and feedback from the nova community about how they think runways have been working or not working and lend any suggestions to change or improve as we continue on in the rocky cycle. We decided to try the runways process to increase the chances of core reviewers converging on the same changes and thus increasing reviews and merges on approved blueprint work. As of today, we have 69 blueprints approved and 28 blueprints completed, we just passed r-2 June 7 and r-3 is July 26 and rc1 is August 9 [2]. Do people feel like they've been receiving more review on their blueprints? Does it seem like we're completing more blueprints earlier? Is there feedback or suggestions for change that you can share? Looking at the Queens burndown chart from Matt [3] we had 11 completed bps at Queens milestone 2. So having 28 completed bps at R-2 means a really nice improvement on our bp completion rate. I think the runaways process contributed to this improvement. Did runaway solve the problem that not every equally ready patch gets equal attention from reviewers? Clearly not. But I don't think this would be a realistic goal for runaways. I suggest that in the future we continue the runaway process but we also revive the priority setting process. Before runaways we had 3-4 bps agreed as priority work for a given cycle. I think we had this 3-4 bps in our head for Rocky as well we just did not write them down. I feel this causes misunderstanding about priories, like: a) does reviewer X has the same 3-4 bps in her/his head with priority as in mine? b) does something that I think part of the 3-4 priority bps has more importance than what is in a runaway slot? Of course when I select what to review priority is only a single factor and there are others, like: * Do I have knowledge about the feature? (Did I review the related spec? Do I have knowledge in the domain or in the impacted code path?) * Is it seems easy to review? (e.g. low complexity feature, small patches, well written commit message) * Is it something that feels important to me, regardless of priority set by the community. (e.g. Do I get frequent company internal questions about the feature? Do I have another feature that depends on this feature as prerequisite work?) So during the cycle it happened that I selected patches to review even if they wasn't in a runaway slot and ignored some patches from the runaway slots. Cheers, gibi [3] https://docs.google.com/spreadsheets/d/e/2PACX-1vRh5glbJ44-Ru2iARidNRa7uFfn2yjiRPjHIEQOc3Fjp5YDAlcMmXkYAEFW0WNhALl010T4rzyChuO9/pubhtml?gid=128173249&single=true Thanks all, -melanie [1] https://etherpad.openstack.org/p/nova-runways-rocky [2] https://wiki.openstack.org/wiki/Nova/Rocky_Release_Schedule __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] review runways check-in and feedback
> While I have tried to review a few of the runway-slotted efforts, I > have gotten burned out on a number of them. Other runway-slotted > efforts, I simply don't care enough about or once I've seen some of > the code, simply can't bring myself to review it (sorry, just being > honest). I have the same feeling, although I have reviewed a lot of things I wouldn't have otherwise as a result of them being in the runway. I spent a bunch of time early on with the image signing stuff, which I think was worthwhile, although at this point I'm a bit worn out on it. That's not the fault of runways though. > Is your concern that placement stuff is getting unfair attention since > many of the patch series aren't in the runways? Or is your concern > that you'd like to see *more* core reviews on placement stuff outside > of the usual placement-y core reviewers (you, me, Alex, Eric, Gibi and > Dan)? I think placement has been getting a bit of a free ride, with constant review and insulation from the runway process. However, I don't think that we can stop progress on that effort while we circle around, and the subteam/group of people that focus on placement already has a lot of supporting cores already. So, it's cheating a little bit, but we always said that we're not going to tell cores *not* to review something unless it is in a runway and pragmatially I think it's probably the right thing to do for placement. >> Having said that, it's clear from the list of things in the runways >> etherpad that there are some lower priority efforts that have been >> completed probably because they leveraged runways (there are a few >> xenapi blueprints for example, and the powervm driver changes). > > Wasn't that kind of the point of the runways, though? To enable "lower > priority" efforts to have a chance at getting reviews? Or are you just > stating here the apparent success of that effort? It was, and I think it has worked well for that for several things. The image signing stuff got more review in its first runway slot than it has in years I think. Overall, I don't think we're worse off with runways than we were before it. I think that some things that will get attention regardless are still progressing. I think that some things that are far off on the fringe are still getting ignored. I think that for the huge bulk of things in the middle of those two, runways has helped focus review on specific efforts and thus increased the throughput there. For a first attempt, I'd call that a success. I think maybe a little more monitoring of the review rate of things in the runways and some gentle prodding of people to look at ones that are burning time and not seeing much review would maybe improve things a bit. --Dan __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] review runways check-in and feedback
On 06/13/2018 05:33 PM, Matt Riedemann wrote: On 6/13/2018 3:33 PM, melanie witt wrote: We've been experimenting with a new process this cycle, Review Runways [1] and we're about at the middle of the cycle now as we had the r-2 milestone last week June 7. I wanted to start a thread and gather thoughts and feedback from the nova community about how they think runways have been working or not working and lend any suggestions to change or improve as we continue on in the rocky cycle. We decided to try the runways process to increase the chances of core reviewers converging on the same changes and thus increasing reviews and merges on approved blueprint work. As of today, we have 69 blueprints approved and 28 blueprints completed, we just passed r-2 June 7 and r-3 is July 26 and rc1 is August 9 [2]. Do people feel like they've been receiving more review on their blueprints? Does it seem like we're completing more blueprints earlier? Is there feedback or suggestions for change that you can share? Lots of cores are not reviewing stuff in the current runways slots, which defeats the purpose of runways for the most part if the majority of the core team aren't going to review what's in a slot. I know I don't review a ton of stuff like you or Eric, but I just can't any more. It's too much for me to handle. While I have tried to review a few of the runway-slotted efforts, I have gotten burned out on a number of them. Other runway-slotted efforts, I simply don't care enough about or once I've seen some of the code, simply can't bring myself to review it (sorry, just being honest). I like the *concept* of the runways, though. It's good to have a focusing agent to direct reviewer attention to things that are "ready" for final review. Despite this focusing agent, though, we are still realistically limited by the small size of the Nova core team. I'm not sure there are processes (runways or otherwise) that are going to increase the velocity of merging code [1] unless we increase the size of the core team. It's not like we don't look for new core additions and attempt to identify folks that would be good cores and try to help them. We *do* do this. The issue is that Nova is big, scary, messy, fragile (in many ways), complex and more than any other project (no offense to those other projects) has a virtually *endless* stream of feature requests coming (mostly from vendors, sorry) looking to plug their latest and greatest hardware into the virt world. Until that endless stream of feature requests subsides, we will continue to have these problems. And, for those out there that say "well, Jay, then those vendors will just abandon OpenStack and go to more fertile feature-accepting grounds like k8s!", I say "hey, go for it." Not everything is appropriate to jam into Nova (or OpenStack for that matter). Let k8s deal with the never-ending feature velocity (NFV) and vendor/product-enablement requests. And let them collapse under that weight. [1] I say "increase the velocity of merging code" but keep in mind that Nova *already* merges the most code in all of OpenStack. We merge more code in Nova in a week than some service projects merge in three months. Our rate of code merging in just Nova often rivals larger-scoped monoliths like kubernetes/kubernetes. Lots of people have ready-for-runways blueprint series that aren't queued up in the runways etherpad, and then ask for reviews on those series and I have to tell them, "throw it in the runways queue". I'm not sure if people are thinking subteams need to review series that are ready for wider review first, but especially for the placement stuff, I think those things need to be slotted up if they are ready. I can work with Eric to make sure placement patch series (for the required ones at least that are holding up other work) are queued up properly for runways. That said, I don't feel we are suffering from a lack of reviews in placement-land. Is your concern that placement stuff is getting unfair attention since many of the patch series aren't in the runways? Or is your concern that you'd like to see *more* core reviews on placement stuff outside of the usual placement-y core reviewers (you, me, Alex, Eric, Gibi and Dan)? Having said that, it's clear from the list of things in the runways etherpad that there are some lower priority efforts that have been completed probably because they leveraged runways (there are a few xenapi blueprints for example, and the powervm driver changes). Wasn't that kind of the point of the runways, though? To enable "lower priority" efforts to have a chance at getting reviews? Or are you just stating here the apparent success of that effort? Best, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
Re: [openstack-dev] [nova] review runways check-in and feedback
On 6/13/2018 3:33 PM, melanie witt wrote: We've been experimenting with a new process this cycle, Review Runways [1] and we're about at the middle of the cycle now as we had the r-2 milestone last week June 7. I wanted to start a thread and gather thoughts and feedback from the nova community about how they think runways have been working or not working and lend any suggestions to change or improve as we continue on in the rocky cycle. We decided to try the runways process to increase the chances of core reviewers converging on the same changes and thus increasing reviews and merges on approved blueprint work. As of today, we have 69 blueprints approved and 28 blueprints completed, we just passed r-2 June 7 and r-3 is July 26 and rc1 is August 9 [2]. Do people feel like they've been receiving more review on their blueprints? Does it seem like we're completing more blueprints earlier? Is there feedback or suggestions for change that you can share? Lots of cores are not reviewing stuff in the current runways slots, which defeats the purpose of runways for the most part if the majority of the core team aren't going to review what's in a slot. Lots of people have ready-for-runways blueprint series that aren't queued up in the runways etherpad, and then ask for reviews on those series and I have to tell them, "throw it in the runways queue". I'm not sure if people are thinking subteams need to review series that are ready for wider review first, but especially for the placement stuff, I think those things need to be slotted up if they are ready. Just because it's in the queue doesn't mean interested parties can't review it. I've seen things in queue get merged before they hit a slot, and that's fine. I personally would also like to see stuff in the queue so I can get a better idea of what is being worked on and what's ready, especially as we wind down into the 3rd milestone and we're going to start crunching for major deliverables that aren't yet done. Speaking just for myself, I've got a bit of anxiety going on right now because I have a feeling we have several major efforts that still need to get done for Rocky and they aren't getting the proper focus from the majority of the team (the nested resource provider migration stuff and handling a down cell are the major ones on my list). Having said that, it's clear from the list of things in the runways etherpad that there are some lower priority efforts that have been completed probably because they leveraged runways (there are a few xenapi blueprints for example, and the powervm driver changes). -- Thanks, Matt __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] review runways check-in and feedback
Howdy everyone, We've been experimenting with a new process this cycle, Review Runways [1] and we're about at the middle of the cycle now as we had the r-2 milestone last week June 7. I wanted to start a thread and gather thoughts and feedback from the nova community about how they think runways have been working or not working and lend any suggestions to change or improve as we continue on in the rocky cycle. We decided to try the runways process to increase the chances of core reviewers converging on the same changes and thus increasing reviews and merges on approved blueprint work. As of today, we have 69 blueprints approved and 28 blueprints completed, we just passed r-2 June 7 and r-3 is July 26 and rc1 is August 9 [2]. Do people feel like they've been receiving more review on their blueprints? Does it seem like we're completing more blueprints earlier? Is there feedback or suggestions for change that you can share? Thanks all, -melanie [1] https://etherpad.openstack.org/p/nova-runways-rocky [2] https://wiki.openstack.org/wiki/Nova/Rocky_Release_Schedule __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
