Re: [openstack-dev] [Nova] Blueprint review process
Stefano Maffulli wrote: I have the feeling we keep going back to communicating expectations to new participants to the community. Are we putting too much emphasis on new commits and too little on new reviews? What do you think if from now on the weekly newsletter would mention the new first time reviewers? We have that report ready: http://activity.openstack.org/data/display/OPNSTK2/New+Contributors+First+Review+-+Last+30+Days What other sort of other incentive do you think we can give to +1ers, the reviewers that without being core, can make life so much easier and shorten the time to get the +2s? Mentioning reviews in our dev documentation (as suggested by Dan Berrange in another thread) should be the first step. You could mention first-time reviewers on an equal footing with first-time authors on the weekly newsletter, but it could quickly get noisy. -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Blueprint review process
On 10/29/2013 07:14 PM, Tom Fifield wrote: On 30/10/13 07:58, Russell Bryant wrote: On 10/29/2013 04:24 PM, Stefano Maffulli wrote: On 10/28/2013 10:28 AM, Russell Bryant wrote: 2) Setting clearer expectations. Since we have so many blueprints for Nova, I feel it's very important to accurately set expectations for how the priority of different projects compare. In the last cycle, priorities were mainly subjectively set by me. Setting priorities based on what reviewers are willing to spend time on is a more accurate reflection of the likelihood of a set of changes making it in to the release. I'm all for managing expectations :) I had a conversation with Tom about this and we agreed that there may be a risk that new contributors with not much karma in the project would have a harder time getting their blueprint assigned higher priorities. If a new group proposes a blueprint, they may need to court bp reviewers to convince them to dedicate attention to their first bp. The risk is that the reviewers of Blueprints risk of becoming a sort of gatekeeper, or what other projects call 'committers'. I think this is a concrete risk, it exists but I don't know if it's possible to eliminate it. I don't think we have to eliminate it but we need to manage it to minimize it in order to keep our promise of being 'open' as in open to new contributors, even the ones with low karma. What do you think? I think you're right, but it's actually no different than things have been in the past. It's just blueprints better reflecting how things actually work. However, people that have a proven track record of producing high quality work are going to have an easier time getting attention, because it takes less work overall to get their patches in. With that said, if the blueprint is good, it should get priority based on its merit, even if the submitter has lower karma in the community. Where we seem to hit the most friction is actually when merit alone doesn't grant higher priority (only relevant to a small number of users, for example), and submitter hasn't built up karma, either. Those are the ones that have a hard time, but it's not that surprising. The user committee might be able to help here. Through the user survey, and engagement with communities around the world, they have an idea of what affects what number of users and how. So, how would you feel about giving some priority manipulation abilities to the user committee? :) Abilities, no, but input, of course. If users are screaming for something, then we should absolutely be paying attention to that. But at the end of the day, the priorities still have to be based on where code reviewers are willing to spend their time. FWIW, I love the user survey and use the results to help me think about priorities. -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Blueprint review process
On 30/10/13 17:14, Russell Bryant wrote: On 10/29/2013 07:14 PM, Tom Fifield wrote: On 30/10/13 07:58, Russell Bryant wrote: On 10/29/2013 04:24 PM, Stefano Maffulli wrote: On 10/28/2013 10:28 AM, Russell Bryant wrote: 2) Setting clearer expectations. Since we have so many blueprints for Nova, I feel it's very important to accurately set expectations for how the priority of different projects compare. In the last cycle, priorities were mainly subjectively set by me. Setting priorities based on what reviewers are willing to spend time on is a more accurate reflection of the likelihood of a set of changes making it in to the release. I'm all for managing expectations :) I had a conversation with Tom about this and we agreed that there may be a risk that new contributors with not much karma in the project would have a harder time getting their blueprint assigned higher priorities. If a new group proposes a blueprint, they may need to court bp reviewers to convince them to dedicate attention to their first bp. The risk is that the reviewers of Blueprints risk of becoming a sort of gatekeeper, or what other projects call 'committers'. I think this is a concrete risk, it exists but I don't know if it's possible to eliminate it. I don't think we have to eliminate it but we need to manage it to minimize it in order to keep our promise of being 'open' as in open to new contributors, even the ones with low karma. What do you think? I think you're right, but it's actually no different than things have been in the past. It's just blueprints better reflecting how things actually work. However, people that have a proven track record of producing high quality work are going to have an easier time getting attention, because it takes less work overall to get their patches in. With that said, if the blueprint is good, it should get priority based on its merit, even if the submitter has lower karma in the community. Where we seem to hit the most friction is actually when merit alone doesn't grant higher priority (only relevant to a small number of users, for example), and submitter hasn't built up karma, either. Those are the ones that have a hard time, but it's not that surprising. The user committee might be able to help here. Through the user survey, and engagement with communities around the world, they have an idea of what affects what number of users and how. So, how would you feel about giving some priority manipulation abilities to the user committee? :) Abilities, no, but input, of course. Any practical ideas on the best way to make that work for you? ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Blueprint review process
On 10/30/2013 02:26 AM, Tom Fifield wrote: On 30/10/13 17:14, Russell Bryant wrote: On 10/29/2013 07:14 PM, Tom Fifield wrote: So, how would you feel about giving some priority manipulation abilities to the user committee? :) Abilities, no, but input, of course. Any practical ideas on the best way to make that work for you? How about having someone drop by the weekly nova meeting any time they would like to discuss something? -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Blueprint review process
Stefano Maffulli wrote: On 10/28/2013 10:28 AM, Russell Bryant wrote: 2) Setting clearer expectations. Since we have so many blueprints for Nova, I feel it's very important to accurately set expectations for how the priority of different projects compare. In the last cycle, priorities were mainly subjectively set by me. Setting priorities based on what reviewers are willing to spend time on is a more accurate reflection of the likelihood of a set of changes making it in to the release. I'm all for managing expectations :) I had a conversation with Tom about this and we agreed that there may be a risk that new contributors with not much karma in the project would have a harder time getting their blueprint assigned higher priorities. If a new group proposes a blueprint, they may need to court bp reviewers to convince them to dedicate attention to their first bp. The risk is that the reviewers of Blueprints risk of becoming a sort of gatekeeper, or what other projects call 'committers'. I think this is a concrete risk, it exists but I don't know if it's possible to eliminate it. I don't think we have to eliminate it but we need to manage it to minimize it in order to keep our promise of being 'open' as in open to new contributors, even the ones with low karma. What do you think? Two remarks: (1) Rating blueprints low priority doesn't mean they won't make it. It means we have no idea if they will make it. Maybe the proposer doesn't have a proven track record of hitting promised deadlines, maybe we don't have core reviewers who signed up to review the work even before it was proposed. Looking back to the havana cycle you can see that a *lot* of Low blueprints made it. It's just hard to predict that they would at the beginning of the cycle. Priority is not really the right word for it. Certainty would be better. (2) About creating gatekeepers like committers: one key difference is that anyone can participate in code reviews, so the gatekeepers are actually an open group. That makes quite a lot of difference. If the process doesn't specialcase core reviewers too much and we have a good history of objectively promoting people with lots of reviews to core reviewer status, we should be good. -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Blueprint review process
On Wed 30 Oct 2013 03:29:32 AM PDT, Thierry Carrez wrote: (1) Rating blueprints low priority doesn't mean they won't make it. It means we have no idea if they will make it. Maybe the proposer doesn't have a proven track record of hitting promised deadlines, maybe we don't have core reviewers who signed up to review the work even before it was proposed. Looking back to the havana cycle you can see that a *lot* of Low blueprints made it. It's just hard to predict that they would at the beginning of the cycle. Priority is not really the right word for it. Certainty would be better. Or it's evil twin, Risk :) (2) About creating gatekeepers like committers: one key difference is that anyone can participate in code reviews, so the gatekeepers are actually an open group. That makes quite a lot of difference. If the process doesn't specialcase core reviewers too much and we have a good history of objectively promoting people with lots of reviews to core reviewer status, we should be good. I have the feeling we keep going back to communicating expectations to new participants to the community. Are we putting too much emphasis on new commits and too little on new reviews? What do you think if from now on the weekly newsletter would mention the new first time reviewers? We have that report ready: http://activity.openstack.org/data/display/OPNSTK2/New+Contributors+First+Review+-+Last+30+Days What other sort of other incentive do you think we can give to +1ers, the reviewers that without being core, can make life so much easier and shorten the time to get the +2s? /stef -- Ask and answer questions on https://ask.openstack.org ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Blueprint review process
On 28 October 2013 15:39, Anne Gentle a...@openstack.org wrote: On Mon, Oct 28, 2013 at 9:24 AM, John Garbutt j...@johngarbutt.com wrote: Here is a really bad attempt at codifying what I think about features vs bugs: 1) If its a new API call, or a change in behaviour, or a new config setting, its a feature 2) If its just refactoring or just adding tests, it is neither a feature or a bug 4) A bug is a change to ensure the system operates as expected by the current documentation This line is the only one I have a little bit of concern with when looking across all projects. We just have to get better at documentation if we're going to make the docs the measure to log bugs against a project. John, your docs are really on target here, but I fear other projects would struggle to set expectations for how something is supposed to work. For example I don't think Hyper-V is documented much at all. So just be careful with this one, use good judgement, and keep this in mind when looking for docs to write. Yes very good point. I was/am in two minds about that: * Part of me wants to say: 5) if there is no documentation, it needs a blueprint, so we can add some. * The other part of me want's to say: as expected by the related blueprint specification, documentation and current user expectations. I am not sure which is best. 3) A bug should be changed to a feature if it matches case (1) If we don't approve the blueprint first, we may end up not having enough information to create the required documentation, so I vote we enforce that a blueprint should be approved before we merge code. Getting a blueprint approved as low priority, should be quicker and easier than getting the associated code approved. Granted that might not be the case today, but we need to fix that. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Blueprint review process
John Garbutt wrote: On 25 October 2013 11:52, Nikola Đipanov ndipa...@redhat.com wrote: I don't have the numbers but I have a feeling that what happened in Havana was that a lot of blueprints slipped until the time for feature freeze. Reviewers thought it was a worthwile feature at that point (this was, I feel, when *actual* blueprint reviews are done - whatever the process says. It's natural too - once the code is there so much more is clear) and wanted to get it in - but it was late in the cycle so we ended up accepting things that could have been done better. Maybe we need a clarification around the priority, something like: * the priority applies only to the target milestone when the priority was agreed * should the blueprint move to a new milestone, the priority should be reset to low priority We should definitely revise priority when a blueprint slips, I'm just not sure there is value in automatically resetting those to Low. I am just wondering about the best way to communicate the revisiting of all the priorities at the beginning of every milestone. I want a way of saying: * reviewers only sign up for a single milestone * if you slip, you (the blueprint developer) need to go make your case again (to raise the prioirty above low) * in most cases the original reviewers will still have bandwidth to help, so ask them first * in many cases the reviewers may have already signed up for the next milestone and re-priortiesed the blueprint for you * but understand you are likely to have a lower priority after the slip, and they could all say no, leaving you as low priority I picked low rather than none because there is no reason to re-approve the blueprint. If it was sane before, its probably still OK, in most cases. Priorities are used to convey how critical a feature is to a given development cycle / release. When people look at our roadmap, they can assume that Essential stuff is more likely to make it than Low stuff. This is in turn reflected in weekly release status meetings where I nag about Essential/High stuff a lot, and I happily ignore Low stuff. We can't just reset Essential stuff to Low if it slips. It will likely stay Essential, it may drop to High, it could go to Low (but that sounds unlikely), it may be deferred. In the (hopefully rare) latter case, we should communicate that and why we did it to our community, so that they can recalibrate their expectations about what will probably be in the release. I agree. I am just wondering about the best way to communicate the cost of slipping. John ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Blueprint review process
On 10/28/2013 10:28 AM, Russell Bryant wrote: 2) Setting clearer expectations. Since we have so many blueprints for Nova, I feel it's very important to accurately set expectations for how the priority of different projects compare. In the last cycle, priorities were mainly subjectively set by me. Setting priorities based on what reviewers are willing to spend time on is a more accurate reflection of the likelihood of a set of changes making it in to the release. I'm all for managing expectations :) I had a conversation with Tom about this and we agreed that there may be a risk that new contributors with not much karma in the project would have a harder time getting their blueprint assigned higher priorities. If a new group proposes a blueprint, they may need to court bp reviewers to convince them to dedicate attention to their first bp. The risk is that the reviewers of Blueprints risk of becoming a sort of gatekeeper, or what other projects call 'committers'. I think this is a concrete risk, it exists but I don't know if it's possible to eliminate it. I don't think we have to eliminate it but we need to manage it to minimize it in order to keep our promise of being 'open' as in open to new contributors, even the ones with low karma. What do you think? We don't really have a specific system for filing feature requests. yep, thanks, this is a different conversation to have. Let's focus on blueprint review first. Thanks, /stef -- Ask and answer questions on https://ask.openstack.org ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Blueprint review process
On 10/29/2013 04:24 PM, Stefano Maffulli wrote: On 10/28/2013 10:28 AM, Russell Bryant wrote: 2) Setting clearer expectations. Since we have so many blueprints for Nova, I feel it's very important to accurately set expectations for how the priority of different projects compare. In the last cycle, priorities were mainly subjectively set by me. Setting priorities based on what reviewers are willing to spend time on is a more accurate reflection of the likelihood of a set of changes making it in to the release. I'm all for managing expectations :) I had a conversation with Tom about this and we agreed that there may be a risk that new contributors with not much karma in the project would have a harder time getting their blueprint assigned higher priorities. If a new group proposes a blueprint, they may need to court bp reviewers to convince them to dedicate attention to their first bp. The risk is that the reviewers of Blueprints risk of becoming a sort of gatekeeper, or what other projects call 'committers'. I think this is a concrete risk, it exists but I don't know if it's possible to eliminate it. I don't think we have to eliminate it but we need to manage it to minimize it in order to keep our promise of being 'open' as in open to new contributors, even the ones with low karma. What do you think? I think you're right, but it's actually no different than things have been in the past. It's just blueprints better reflecting how things actually work. However, people that have a proven track record of producing high quality work are going to have an easier time getting attention, because it takes less work overall to get their patches in. With that said, if the blueprint is good, it should get priority based on its merit, even if the submitter has lower karma in the community. Where we seem to hit the most friction is actually when merit alone doesn't grant higher priority (only relevant to a small number of users, for example), and submitter hasn't built up karma, either. Those are the ones that have a hard time, but it's not that surprising. -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Blueprint review process
On 30/10/13 07:58, Russell Bryant wrote: On 10/29/2013 04:24 PM, Stefano Maffulli wrote: On 10/28/2013 10:28 AM, Russell Bryant wrote: 2) Setting clearer expectations. Since we have so many blueprints for Nova, I feel it's very important to accurately set expectations for how the priority of different projects compare. In the last cycle, priorities were mainly subjectively set by me. Setting priorities based on what reviewers are willing to spend time on is a more accurate reflection of the likelihood of a set of changes making it in to the release. I'm all for managing expectations :) I had a conversation with Tom about this and we agreed that there may be a risk that new contributors with not much karma in the project would have a harder time getting their blueprint assigned higher priorities. If a new group proposes a blueprint, they may need to court bp reviewers to convince them to dedicate attention to their first bp. The risk is that the reviewers of Blueprints risk of becoming a sort of gatekeeper, or what other projects call 'committers'. I think this is a concrete risk, it exists but I don't know if it's possible to eliminate it. I don't think we have to eliminate it but we need to manage it to minimize it in order to keep our promise of being 'open' as in open to new contributors, even the ones with low karma. What do you think? I think you're right, but it's actually no different than things have been in the past. It's just blueprints better reflecting how things actually work. However, people that have a proven track record of producing high quality work are going to have an easier time getting attention, because it takes less work overall to get their patches in. With that said, if the blueprint is good, it should get priority based on its merit, even if the submitter has lower karma in the community. Where we seem to hit the most friction is actually when merit alone doesn't grant higher priority (only relevant to a small number of users, for example), and submitter hasn't built up karma, either. Those are the ones that have a hard time, but it's not that surprising. The user committee might be able to help here. Through the user survey, and engagement with communities around the world, they have an idea of what affects what number of users and how. So, how would you feel about giving some priority manipulation abilities to the user committee? :) Regards, Tom ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Blueprint review process
On 25 October 2013 11:52, Nikola Đipanov ndipa...@redhat.com wrote: On 23/10/13 17:33, Russell Bryant wrote: 4) Blueprint Prioritization I would like to do a better job of using priorities in Icehouse. The priority field services a couple of purposes: - helps reviewers prioritize their time - helps set expectations for the submitter for how reviewing this work stacks up against other things In the last meeting we discussed an idea that I think is worth trying at least for icehouse-1 to see if we like it or not. The idea is that *every* blueprint starts out at a Low priority, which means best effort, but no promises. For a blueprint to get prioritized higher, it should have 2 nova-core members signed up to review the resulting code. I am +1 the updated process. On 25 October 2013 11:52, Nikola Đipanov ndipa...@redhat.com wrote: I don't have the numbers but I have a feeling that what happened in Havana was that a lot of blueprints slipped until the time for feature freeze. Reviewers thought it was a worthwile feature at that point (this was, I feel, when *actual* blueprint reviews are done - whatever the process says. It's natural too - once the code is there so much more is clear) and wanted to get it in - but it was late in the cycle so we ended up accepting things that could have been done better. Maybe we need a clarification around the priority, something like: * the priority applies only to the target milestone when the priority was agreed * should the blueprint move to a new milestone, the priority should be reset to low priority It would be good to levarage the blueprint process to make people post code as soon as possible IMHO. How about making posted code a pre-req for core reviewers to sign up for them? Thoughts? Hmm, I like the idea of encouraging people to submit blueprints, and get them reviewed, before starting to code. If we can help resolve differences at the design phase, it should help make the code review a much happier experience! John ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Blueprint review process
On 25 October 2013 10:18, Joe Gordon joe.gord...@gmail.com wrote: On Oct 24, 2013 9:14 PM, Robert Collins robe...@robertcollins.net wrote: On 24 October 2013 04:33, Russell Bryant rbry...@redhat.com wrote: Greetings, At the last Nova meeting we started talking about some updates to the Nova blueprint process for the Icehouse cycle. I had hoped we could talk about and finalize this in a Nova design summit session on Nova Project Structure and Process [1], but I think we need to push forward on finalizing this as soon as possible so that it doesn't block current work being done. Cool Here is a first cut at the process. Let me know what you think is missing or should change. I'll get the result of this thread posted on the wiki. 1) Proposing a Blueprint Proposing a blueprint for Nova is not much different than other projects. You should follow the instructions here: https://wiki.openstack.org/wiki/Blueprints The particular important step that seems to be missed by most is: Once it is ready for PTL review, you should set: Milestone: Which part of the release cycle you think your work will be proposed for merging. That is really important. Due to the volume of Nova blueprints, it probably will not be seen until you do this. The other thing I'm seeing some friction on is 'significant features' : it sometimes feels like folk are filing blueprints for everything that isn't 'the code crashed' style problems, and while I appreciate folk wanting to work within the system, blueprints are a heavyweight tool, primarily suited for things that require significant coordination. 2) Blueprint Review Team Ensuring blueprints get reviewed is one of the responsibilities of the PTL. However, due to the volume of Nova blueprints, it's not practical for me to do it alone. A team of people (nova-drivers) [2], a subset of nova-core, will be doing blueprint reviews. Why a subset of nova-core? With nova-core defined as 'knows the code well *AND* reviews a lot', I can see that those folk are in a position to spot a large class of design defects. However, there are plenty of folk with expertise in e.g. SOA, operations, deployment @ scale, who are not nova-core but who will spot plenty of issues. Is there some way they can help out? By having more people reviewing blueprints, we can do a more thorough job and have a higher quality result. Note that even though there is a nova-drivers team, *everyone* is encouraged to participate in the review process by providing feedback on the mailing list. I'm not sure about this bit here: blueprints don't have the spec content, usually thats in an etherpad; etherpads are editable by everyone - wouldn't it be better to keep the conversation together? I guess part of my concern here comes back to the (ab)use of blueprints for shallow features. 3) Blueprint Review Criteria Here are some things that the team reviewing blueprints should look for: The blueprint ... - is assigned to the person signing up to do the work - has been targeted to the milestone when the code is planned to be completed - is an appropriate feature for Nova. This means it fits with the vision for Nova and OpenStack overall. This is obviously very subjective, but the result should represent consensus. - includes enough detail to be able to complete an initial design review before approving the blueprint. In many cases, the design review may result in a discussion on the mailing list to work through details. A link to this discussion should be left in the whiteboard of the blueprint for reference. This initial design review should be completed before the blueprint is approved. - includes information that describes the user impact (or lack of). Between the blueprint and text that comes with the DocImpact flag [3] in commits, the docs team should have *everything* they need to thoroughly document the feature. I'd like to add: - has an etherpad with the design (the blueprint summary has no markup and is a poor place for capturing the design). Once the review has been complete, the blueprint should be marked as approved and the priority should be set. A set priority is how we know from the blueprint list which ones have already been reviewed. 4) Blueprint Prioritization I would like to do a better job of using priorities in Icehouse. The priority field services a couple of purposes: - helps reviewers prioritize their time - helps set expectations for the submitter for how reviewing this work stacks up against other things In the last meeting we discussed an idea that I think is worth trying at least for icehouse-1 to see if we like it or not. The idea is that *every* blueprint starts out at a Low priority, which means best effort, but no promises. For a blueprint to get
Re: [openstack-dev] [Nova] Blueprint review process
On 10/23/2013 08:33 AM, Russell Bryant wrote: At the last Nova meeting we started talking about some updates to the Nova blueprint process for the Icehouse cycle. I had hoped we could talk about and finalize this in a Nova design summit session on Nova Project Structure and Process [1], but I think we need to push forward on finalizing this as soon as possible so that it doesn't block current work being done. I understand the need for speed here and I would like to help make sure that any change is effectively communicated to the wider community at this very busy time of the year. Since it's very dangerous to assume that everybody reads the mailing list, I would suggest writing a blog post on openstack.org/blog and a feature in the weekly newsletter is the bare minimum we can do. I'll nag you on IRC if I have questions. I think I have most of the information on this message except it's not clear to me why you are proposing to review the process: can you sumamrize please? 2) Blueprint Review Team [...] Do you think users can get involved at this point to give their input? Or where would you see users getting the chance to give feedback about features they'd need to be implemented? thanks, Stef -- Ask and answer questions on https://ask.openstack.org ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Blueprint review process
On Mon, Oct 28, 2013 at 9:24 AM, John Garbutt j...@johngarbutt.com wrote: On 25 October 2013 10:18, Joe Gordon joe.gord...@gmail.com wrote: On Oct 24, 2013 9:14 PM, Robert Collins robe...@robertcollins.net wrote: On 24 October 2013 04:33, Russell Bryant rbry...@redhat.com wrote: Greetings, At the last Nova meeting we started talking about some updates to the Nova blueprint process for the Icehouse cycle. I had hoped we could talk about and finalize this in a Nova design summit session on Nova Project Structure and Process [1], but I think we need to push forward on finalizing this as soon as possible so that it doesn't block current work being done. Cool Here is a first cut at the process. Let me know what you think is missing or should change. I'll get the result of this thread posted on the wiki. 1) Proposing a Blueprint Proposing a blueprint for Nova is not much different than other projects. You should follow the instructions here: https://wiki.openstack.org/wiki/Blueprints The particular important step that seems to be missed by most is: Once it is ready for PTL review, you should set: Milestone: Which part of the release cycle you think your work will be proposed for merging. That is really important. Due to the volume of Nova blueprints, it probably will not be seen until you do this. The other thing I'm seeing some friction on is 'significant features' : it sometimes feels like folk are filing blueprints for everything that isn't 'the code crashed' style problems, and while I appreciate folk wanting to work within the system, blueprints are a heavyweight tool, primarily suited for things that require significant coordination. 2) Blueprint Review Team Ensuring blueprints get reviewed is one of the responsibilities of the PTL. However, due to the volume of Nova blueprints, it's not practical for me to do it alone. A team of people (nova-drivers) [2], a subset of nova-core, will be doing blueprint reviews. Why a subset of nova-core? With nova-core defined as 'knows the code well *AND* reviews a lot', I can see that those folk are in a position to spot a large class of design defects. However, there are plenty of folk with expertise in e.g. SOA, operations, deployment @ scale, who are not nova-core but who will spot plenty of issues. Is there some way they can help out? By having more people reviewing blueprints, we can do a more thorough job and have a higher quality result. Note that even though there is a nova-drivers team, *everyone* is encouraged to participate in the review process by providing feedback on the mailing list. I'm not sure about this bit here: blueprints don't have the spec content, usually thats in an etherpad; etherpads are editable by everyone - wouldn't it be better to keep the conversation together? I guess part of my concern here comes back to the (ab)use of blueprints for shallow features. 3) Blueprint Review Criteria Here are some things that the team reviewing blueprints should look for: The blueprint ... - is assigned to the person signing up to do the work - has been targeted to the milestone when the code is planned to be completed - is an appropriate feature for Nova. This means it fits with the vision for Nova and OpenStack overall. This is obviously very subjective, but the result should represent consensus. - includes enough detail to be able to complete an initial design review before approving the blueprint. In many cases, the design review may result in a discussion on the mailing list to work through details. A link to this discussion should be left in the whiteboard of the blueprint for reference. This initial design review should be completed before the blueprint is approved. - includes information that describes the user impact (or lack of). Between the blueprint and text that comes with the DocImpact flag [3] in commits, the docs team should have *everything* they need to thoroughly document the feature. I'd like to add: - has an etherpad with the design (the blueprint summary has no markup and is a poor place for capturing the design). Once the review has been complete, the blueprint should be marked as approved and the priority should be set. A set priority is how we know from the blueprint list which ones have already been reviewed. 4) Blueprint Prioritization I would like to do a better job of using priorities in Icehouse. The priority field services a couple of purposes: - helps reviewers prioritize their time - helps set expectations for the submitter for how reviewing this work stacks up against other things In the last meeting we discussed an idea that I think is worth
Re: [openstack-dev] [Nova] Blueprint review process
John Garbutt wrote: On 25 October 2013 11:52, Nikola Đipanov ndipa...@redhat.com wrote: I don't have the numbers but I have a feeling that what happened in Havana was that a lot of blueprints slipped until the time for feature freeze. Reviewers thought it was a worthwile feature at that point (this was, I feel, when *actual* blueprint reviews are done - whatever the process says. It's natural too - once the code is there so much more is clear) and wanted to get it in - but it was late in the cycle so we ended up accepting things that could have been done better. Maybe we need a clarification around the priority, something like: * the priority applies only to the target milestone when the priority was agreed * should the blueprint move to a new milestone, the priority should be reset to low priority We should definitely revise priority when a blueprint slips, I'm just not sure there is value in automatically resetting those to Low. Priorities are used to convey how critical a feature is to a given development cycle / release. When people look at our roadmap, they can assume that Essential stuff is more likely to make it than Low stuff. This is in turn reflected in weekly release status meetings where I nag about Essential/High stuff a lot, and I happily ignore Low stuff. We can't just reset Essential stuff to Low if it slips. It will likely stay Essential, it may drop to High, it could go to Low (but that sounds unlikely), it may be deferred. In the (hopefully rare) latter case, we should communicate that and why we did it to our community, so that they can recalibrate their expectations about what will probably be in the release. Regards, -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Blueprint review process
On 29 October 2013 03:24, John Garbutt j...@johngarbutt.com wrote: I based that on this statement, which I think sums it up well If the patch implements a feature, it should reference a blueprint. The blueprint should be approved before the patch is merged https://wiki.openstack.org/wiki/ReviewChecklist This does raise the question of what is consisted a feature though. Here is a really bad attempt at codifying what I think about features vs bugs: 1) If its a new API call, or a change in behaviour, or a new config setting, its a feature 2) If its just refactoring or just adding tests, it is neither a feature or a bug 4) A bug is a change to ensure the system operates as expected by the current documentation 3) A bug should be changed to a feature if it matches case (1) If we don't approve the blueprint first, we may end up not having enough information to create the required documentation, so I vote we enforce that a blueprint should be approved before we merge code. Getting a blueprint approved as low priority, should be quicker and easier than getting the associated code approved. Granted that might not be the case today, but we need to fix that. I could certainly follow that algorithm, but the implication is that all changes in behaviour are meant to go through careful design review. Right now that certainly doesn't happen - and the blueprint page also says 'major feature'. The algorithm you've put forward also suggests that a blueprint is never needed unless there is a new API call/change in behaviour/config setting - and I'd disagree with that, because design review and approval is useful in major works (such as behaviour preserving refactorings that change the DB schema/message queue utilisation etc - not externally visible but still important to get right). - I think 'bug vs feature' is the wrong language for framing the problem we are trying to solve. Here are my assumptions: - We have rigorous code review by folk who are a) familiar with the domain b) familiar with the skeletons and c) familiar with project history This will catch [most] poor works, whether large or small, at review time. - we want to respect the effort of our contributors and offer a means to steer their work into good designs and avoid problematic designs before significant development effort is invested. [Lets define significant as 2 days coding]. - we want to be able to say record that we said 'no' to some ideas and reference that if they come up again - the same people doing code review are those doing design review, more or less - latency on design review will be worse than that on code review (because written code is inventory, and we should be aiming to land it and avoid rebase hell) Any minor development effort - whether it includes a new config setting, API call, or new behaviour - doesn't run the risk of significant investment, so I think it's entirely appropriate to catch that through code review. Catching it through code review will have less latency that waiting for design review, and code review includes design review anyway. Any major development effort - whether it's a large scale refactoring of unit tests, adding new functionality for users, splitting an existing thing out, adding a new backend - *anything* that is more than [significant effort cutoff point] - is something that will benefit from design review: failing to get design review increases the chance of bad things: - the patch being rejected as a bad idea - the approach needing rework [vs code level 'this would be better' requests] - reviewers disagreeing about the design and stalling the project Design review addresses these problems by a) identifying bad ideas up front, b) vetting the approach via high level walk throughs with experienced reviewers in the project and c) forming consensus amongst the -core reviews about the upcoming work. So - to me - we should focus on the size of the change, not the nature of the change, when talking about 'does it need a blueprint'. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Blueprint review process
On Oct 24, 2013 9:14 PM, Robert Collins robe...@robertcollins.net wrote: On 24 October 2013 04:33, Russell Bryant rbry...@redhat.com wrote: Greetings, At the last Nova meeting we started talking about some updates to the Nova blueprint process for the Icehouse cycle. I had hoped we could talk about and finalize this in a Nova design summit session on Nova Project Structure and Process [1], but I think we need to push forward on finalizing this as soon as possible so that it doesn't block current work being done. Cool Here is a first cut at the process. Let me know what you think is missing or should change. I'll get the result of this thread posted on the wiki. 1) Proposing a Blueprint Proposing a blueprint for Nova is not much different than other projects. You should follow the instructions here: https://wiki.openstack.org/wiki/Blueprints The particular important step that seems to be missed by most is: Once it is ready for PTL review, you should set: Milestone: Which part of the release cycle you think your work will be proposed for merging. That is really important. Due to the volume of Nova blueprints, it probably will not be seen until you do this. The other thing I'm seeing some friction on is 'significant features' : it sometimes feels like folk are filing blueprints for everything that isn't 'the code crashed' style problems, and while I appreciate folk wanting to work within the system, blueprints are a heavyweight tool, primarily suited for things that require significant coordination. 2) Blueprint Review Team Ensuring blueprints get reviewed is one of the responsibilities of the PTL. However, due to the volume of Nova blueprints, it's not practical for me to do it alone. A team of people (nova-drivers) [2], a subset of nova-core, will be doing blueprint reviews. Why a subset of nova-core? With nova-core defined as 'knows the code well *AND* reviews a lot', I can see that those folk are in a position to spot a large class of design defects. However, there are plenty of folk with expertise in e.g. SOA, operations, deployment @ scale, who are not nova-core but who will spot plenty of issues. Is there some way they can help out? By having more people reviewing blueprints, we can do a more thorough job and have a higher quality result. Note that even though there is a nova-drivers team, *everyone* is encouraged to participate in the review process by providing feedback on the mailing list. I'm not sure about this bit here: blueprints don't have the spec content, usually thats in an etherpad; etherpads are editable by everyone - wouldn't it be better to keep the conversation together? I guess part of my concern here comes back to the (ab)use of blueprints for shallow features. 3) Blueprint Review Criteria Here are some things that the team reviewing blueprints should look for: The blueprint ... - is assigned to the person signing up to do the work - has been targeted to the milestone when the code is planned to be completed - is an appropriate feature for Nova. This means it fits with the vision for Nova and OpenStack overall. This is obviously very subjective, but the result should represent consensus. - includes enough detail to be able to complete an initial design review before approving the blueprint. In many cases, the design review may result in a discussion on the mailing list to work through details. A link to this discussion should be left in the whiteboard of the blueprint for reference. This initial design review should be completed before the blueprint is approved. - includes information that describes the user impact (or lack of). Between the blueprint and text that comes with the DocImpact flag [3] in commits, the docs team should have *everything* they need to thoroughly document the feature. I'd like to add: - has an etherpad with the design (the blueprint summary has no markup and is a poor place for capturing the design). Once the review has been complete, the blueprint should be marked as approved and the priority should be set. A set priority is how we know from the blueprint list which ones have already been reviewed. 4) Blueprint Prioritization I would like to do a better job of using priorities in Icehouse. The priority field services a couple of purposes: - helps reviewers prioritize their time - helps set expectations for the submitter for how reviewing this work stacks up against other things In the last meeting we discussed an idea that I think is worth trying at least for icehouse-1 to see if we like it or not. The idea is that *every* blueprint starts out at a Low priority, which means best effort, but no promises. For a blueprint to get prioritized higher, it should have 2 nova-core members signed up to
Re: [openstack-dev] [Nova] Blueprint review process
On 23/10/13 17:33, Russell Bryant wrote: 4) Blueprint Prioritization I would like to do a better job of using priorities in Icehouse. The priority field services a couple of purposes: - helps reviewers prioritize their time - helps set expectations for the submitter for how reviewing this work stacks up against other things In the last meeting we discussed an idea that I think is worth trying at least for icehouse-1 to see if we like it or not. The idea is that *every* blueprint starts out at a Low priority, which means best effort, but no promises. For a blueprint to get prioritized higher, it should have 2 nova-core members signed up to review the resulting code. All of the mentioned seem like awesome ideas that I +1 wholeheartedly. A comment on this point though. I don't have the numbers but I have a feeling that what happened in Havana was that a lot of blueprints slipped until the time for feature freeze. Reviewers thought it was a worthwile feature at that point (this was, I feel, when *actual* blueprint reviews are done - whatever the process says. It's natural too - once the code is there so much more is clear) and wanted to get it in - but it was late in the cycle so we ended up accepting things that could have been done better. It would be good to levarage the blueprint process to make people post code as soon as possible IMHO. How about making posted code a pre-req for core reviewers to sign up for them? Thoughts? Thanks, N. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Blueprint review process
It would be helpful if you could follow the reply style being used. :-) -Original Message- From: Russell Bryant [mailto:rbry...@redhat.com] Sent: October-24-13 5:08 PM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [Nova] Blueprint review process On 10/24/2013 10:52 AM, Gary Kotton wrote: On 10/24/13 4:46 PM, Dan Smith d...@danplanet.com wrote: In the last meeting we discussed an idea that I think is worth trying at least for icehouse-1 to see if we like it or not. The idea is that *every* blueprint starts out at a Low priority, which means best effort, but no promises. For a blueprint to get prioritized higher, it should have 2 nova-core members signed up to review the resulting code. Huge +1 to this. I'm in favor of the whole plan, but specifically the prioritization piece is very important, IMHO. I too am in favor of the idea. It is just not clear how 2 Nova cores will be signed up. Good point, there was no detail on that. I propose just comments on the blueprint whiteboard. It can be something simple like this to indicate that Dan and I have agreed to review the code for something: nova-core reviewers: russellb, dansmith On 10/24/2013 06:17 PM, Alan Kavanagh wrote: Is this really a viable solution? I believe its more democratic to ensure everyone gets a chance to present the blueprint someone has spent time to write. This was no favoritism or biased view will ever take place and we let the community gauge the interest. I don't really understand. The key to this is that it really doesn't change anything for the end result. It just makes the blueprint list a better reflection of what was already happening. Note that the prioritization changes have nothing to do with whether blueprints are accepted or not. That's a separate issue. That part isn't changing much beyond having more people helping review blueprints and having more explicit blueprint review criteria. -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Blueprint review process
On Wed, Oct 23, 2013 at 4:33 PM, Russell Bryant rbry...@redhat.com wrote: Greetings, At the last Nova meeting we started talking about some updates to the Nova blueprint process for the Icehouse cycle. I had hoped we could talk about and finalize this in a Nova design summit session on Nova Project Structure and Process [1], but I think we need to push forward on finalizing this as soon as possible so that it doesn't block current work being done. Here is a first cut at the process. Let me know what you think is missing or should change. I'll get the result of this thread posted on the wiki. 1) Proposing a Blueprint Proposing a blueprint for Nova is not much different than other projects. You should follow the instructions here: https://wiki.openstack.org/wiki/Blueprints The particular important step that seems to be missed by most is: Once it is ready for PTL review, you should set: Milestone: Which part of the release cycle you think your work will be proposed for merging. That is really important. Due to the volume of Nova blueprints, it probably will not be seen until you do this. 2) Blueprint Review Team Ensuring blueprints get reviewed is one of the responsibilities of the PTL. However, due to the volume of Nova blueprints, it's not practical for me to do it alone. A team of people (nova-drivers) [2], a subset of nova-core, will be doing blueprint reviews. By having more people reviewing blueprints, we can do a more thorough job and have a higher quality result. Note that even though there is a nova-drivers team, *everyone* is encouraged to participate in the review process by providing feedback on the mailing list. 3) Blueprint Review Criteria Here are some things that the team reviewing blueprints should look for: The blueprint ... - is assigned to the person signing up to do the work - has been targeted to the milestone when the code is planned to be completed - is an appropriate feature for Nova. This means it fits with the vision for Nova and OpenStack overall. This is obviously very subjective, but the result should represent consensus. - includes enough detail to be able to complete an initial design review before approving the blueprint. In many cases, the design review may result in a discussion on the mailing list to work through details. A link to this discussion should be left in the whiteboard of the blueprint for reference. This initial design review should be completed before the blueprint is approved. - includes information that describes the user impact (or lack of). Between the blueprint and text that comes with the DocImpact flag [3] in commits, the docs team should have *everything* they need to thoroughly document the feature. Once the review has been complete, the blueprint should be marked as approved and the priority should be set. A set priority is how we know from the blueprint list which ones have already been reviewed. 4) Blueprint Prioritization I would like to do a better job of using priorities in Icehouse. The priority field services a couple of purposes: - helps reviewers prioritize their time - helps set expectations for the submitter for how reviewing this work stacks up against other things In the last meeting we discussed an idea that I think is worth trying at least for icehouse-1 to see if we like it or not. The idea is that *every* blueprint starts out at a Low priority, which means best effort, but no promises. For a blueprint to get prioritized higher, it should have 2 nova-core members signed up to review the resulting code. If we do this, I suspect we may end up with more blueprints at Low, but I also think we'll end up with a more realistic list of blueprints. The reality is if a feature doesn't have reviewers agreeing to do the review, it really is in a best effort, but no promises situation. 5) Blueprint Fall Cleaning Finally, it's about time we do some cleaning of the blueprint backlog. There are a bunch not currently being worked on. I propose that we close out all blueprints not targeted at a release milestone by November 22 (2 weeks after the end of the design summit), with the exception of anything just recently filed and still being drafted. ++ to the entire process, two comments though. * It would be great to get this into a wiki for future reference * We shouldn't merge patches with un-approved blueprints. And when that happens having a wiki page to point to would be great. [1] http://summit.openstack.org/cfp/details/341 [2] https://launchpad.net/~nova-drivers/+members#active [3] http://justwriteclick.com/2013/09/17/openstack-docimpact-flag-walk-through/ -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Blueprint review process
In the last meeting we discussed an idea that I think is worth trying at least for icehouse-1 to see if we like it or not. The idea is that *every* blueprint starts out at a Low priority, which means best effort, but no promises. For a blueprint to get prioritized higher, it should have 2 nova-core members signed up to review the resulting code. Huge +1 to this. I'm in favor of the whole plan, but specifically the prioritization piece is very important, IMHO. --Dan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Blueprint review process
On 10/24/13 4:46 PM, Dan Smith d...@danplanet.com wrote: In the last meeting we discussed an idea that I think is worth trying at least for icehouse-1 to see if we like it or not. The idea is that *every* blueprint starts out at a Low priority, which means best effort, but no promises. For a blueprint to get prioritized higher, it should have 2 nova-core members signed up to review the resulting code. Huge +1 to this. I'm in favor of the whole plan, but specifically the prioritization piece is very important, IMHO. I too am in favor of the idea. It is just not clear how 2 Nova cores will be signed up. --Dan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Blueprint review process
On 10/24/2013 07:43 AM, Joe Gordon wrote: On Wed, Oct 23, 2013 at 4:33 PM, Russell Bryant rbry...@redhat.com mailto:rbry...@redhat.com wrote: Here is a first cut at the process. Let me know what you think is missing or should change. I'll get the result of this thread posted on the wiki. ++ to the entire process, two comments though. * It would be great to get this into a wiki for future reference * We shouldn't merge patches with un-approved blueprints. And when that happens having a wiki page to point to would be great. Yep, I do want to get this on the wiki. I just put it out on the list for comments before making it official. -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Blueprint review process
On 10/24/2013 10:52 AM, Gary Kotton wrote: On 10/24/13 4:46 PM, Dan Smith d...@danplanet.com wrote: In the last meeting we discussed an idea that I think is worth trying at least for icehouse-1 to see if we like it or not. The idea is that *every* blueprint starts out at a Low priority, which means best effort, but no promises. For a blueprint to get prioritized higher, it should have 2 nova-core members signed up to review the resulting code. Huge +1 to this. I'm in favor of the whole plan, but specifically the prioritization piece is very important, IMHO. I too am in favor of the idea. It is just not clear how 2 Nova cores will be signed up. Good point, there was no detail on that. I propose just comments on the blueprint whiteboard. It can be something simple like this to indicate that Dan and I have agreed to review the code for something: nova-core reviewers: russellb, dansmith -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Blueprint review process
On 10/24/13 at 11:07am, Russell Bryant wrote: On 10/24/2013 10:52 AM, Gary Kotton wrote: On 10/24/13 4:46 PM, Dan Smith d...@danplanet.com wrote: In the last meeting we discussed an idea that I think is worth trying at least for icehouse-1 to see if we like it or not. The idea is that *every* blueprint starts out at a Low priority, which means best effort, but no promises. For a blueprint to get prioritized higher, it should have 2 nova-core members signed up to review the resulting code. Huge +1 to this. I'm in favor of the whole plan, but specifically the prioritization piece is very important, IMHO. I too am in favor of the idea. It is just not clear how 2 Nova cores will be signed up. Good point, there was no detail on that. I propose just comments on the blueprint whiteboard. It can be something simple like this to indicate that Dan and I have agreed to review the code for something: nova-core reviewers: russellb, dansmith +1 to everything in Russells original email. But for this point specifically I see it as resulting from conversations amongst Nova developers. If some of us decide that a blueprint is important or very nice to have then we should sign up to help it through. But there's nothing wrong with a low priority blueprint. We may want to communicate that core members don't need to be hunted and recruited for absolutely every blueprint that's proposed. -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Blueprint review process
Russell Bryant wrote: At the last Nova meeting we started talking about some updates to the Nova blueprint process for the Icehouse cycle. I had hoped we could talk about and finalize this in a Nova design summit session on Nova Project Structure and Process [1], but I think we need to push forward on finalizing this as soon as possible so that it doesn't block current work being done. Here is a first cut at the process. Let me know what you think is missing or should change. I'll get the result of this thread posted on the wiki. [...] +1 That's pretty much how I would like every project to handle their blueprints. For smaller projects I guess the 2 core signed up for =Medium blueprints requirement can be handled informally, but the rest is spot-on and should be applicable everywhere. -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Blueprint review process
On 10/24/2013 11:32 AM, Thierry Carrez wrote: Russell Bryant wrote: At the last Nova meeting we started talking about some updates to the Nova blueprint process for the Icehouse cycle. I had hoped we could talk about and finalize this in a Nova design summit session on Nova Project Structure and Process [1], but I think we need to push forward on finalizing this as soon as possible so that it doesn't block current work being done. Here is a first cut at the process. Let me know what you think is missing or should change. I'll get the result of this thread posted on the wiki. [...] +1 That's pretty much how I would like every project to handle their blueprints. For smaller projects I guess the 2 core signed up for =Medium blueprints requirement can be handled informally, but the rest is spot-on and should be applicable everywhere. If that's the case, then I can just work on updating the main Blueprints page [1] with a little bit more detail. I suppose there's not that much missing. In particular, I would add: - notes on blueprint review criteria - the use of -driver teams by some projects to review - some more info on prioriization based on review bandwidth, and nova's specific requirement for reviewer support for a priority Low [1] https://wiki.openstack.org/wiki/Blueprints -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Blueprint review process
On 24 October 2013 04:33, Russell Bryant rbry...@redhat.com wrote: Greetings, At the last Nova meeting we started talking about some updates to the Nova blueprint process for the Icehouse cycle. I had hoped we could talk about and finalize this in a Nova design summit session on Nova Project Structure and Process [1], but I think we need to push forward on finalizing this as soon as possible so that it doesn't block current work being done. Cool Here is a first cut at the process. Let me know what you think is missing or should change. I'll get the result of this thread posted on the wiki. 1) Proposing a Blueprint Proposing a blueprint for Nova is not much different than other projects. You should follow the instructions here: https://wiki.openstack.org/wiki/Blueprints The particular important step that seems to be missed by most is: Once it is ready for PTL review, you should set: Milestone: Which part of the release cycle you think your work will be proposed for merging. That is really important. Due to the volume of Nova blueprints, it probably will not be seen until you do this. The other thing I'm seeing some friction on is 'significant features' : it sometimes feels like folk are filing blueprints for everything that isn't 'the code crashed' style problems, and while I appreciate folk wanting to work within the system, blueprints are a heavyweight tool, primarily suited for things that require significant coordination. 2) Blueprint Review Team Ensuring blueprints get reviewed is one of the responsibilities of the PTL. However, due to the volume of Nova blueprints, it's not practical for me to do it alone. A team of people (nova-drivers) [2], a subset of nova-core, will be doing blueprint reviews. Why a subset of nova-core? With nova-core defined as 'knows the code well *AND* reviews a lot', I can see that those folk are in a position to spot a large class of design defects. However, there are plenty of folk with expertise in e.g. SOA, operations, deployment @ scale, who are not nova-core but who will spot plenty of issues. Is there some way they can help out? By having more people reviewing blueprints, we can do a more thorough job and have a higher quality result. Note that even though there is a nova-drivers team, *everyone* is encouraged to participate in the review process by providing feedback on the mailing list. I'm not sure about this bit here: blueprints don't have the spec content, usually thats in an etherpad; etherpads are editable by everyone - wouldn't it be better to keep the conversation together? I guess part of my concern here comes back to the (ab)use of blueprints for shallow features. 3) Blueprint Review Criteria Here are some things that the team reviewing blueprints should look for: The blueprint ... - is assigned to the person signing up to do the work - has been targeted to the milestone when the code is planned to be completed - is an appropriate feature for Nova. This means it fits with the vision for Nova and OpenStack overall. This is obviously very subjective, but the result should represent consensus. - includes enough detail to be able to complete an initial design review before approving the blueprint. In many cases, the design review may result in a discussion on the mailing list to work through details. A link to this discussion should be left in the whiteboard of the blueprint for reference. This initial design review should be completed before the blueprint is approved. - includes information that describes the user impact (or lack of). Between the blueprint and text that comes with the DocImpact flag [3] in commits, the docs team should have *everything* they need to thoroughly document the feature. I'd like to add: - has an etherpad with the design (the blueprint summary has no markup and is a poor place for capturing the design). Once the review has been complete, the blueprint should be marked as approved and the priority should be set. A set priority is how we know from the blueprint list which ones have already been reviewed. 4) Blueprint Prioritization I would like to do a better job of using priorities in Icehouse. The priority field services a couple of purposes: - helps reviewers prioritize their time - helps set expectations for the submitter for how reviewing this work stacks up against other things In the last meeting we discussed an idea that I think is worth trying at least for icehouse-1 to see if we like it or not. The idea is that *every* blueprint starts out at a Low priority, which means best effort, but no promises. For a blueprint to get prioritized higher, it should have 2 nova-core members signed up to review the resulting code. If we do this, I suspect we may end up with more blueprints at Low, but I also think we'll end up with a more realistic list of blueprints. The
Re: [openstack-dev] [Nova] Blueprint review process
Is this really a viable solution? I believe its more democratic to ensure everyone gets a chance to present the blueprint someone has spent time to write. This was no favoritism or biased view will ever take place and we let the community gauge the interest. /Alan -Original Message- From: Russell Bryant [mailto:rbry...@redhat.com] Sent: October-24-13 5:08 PM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [Nova] Blueprint review process On 10/24/2013 10:52 AM, Gary Kotton wrote: On 10/24/13 4:46 PM, Dan Smith d...@danplanet.com wrote: In the last meeting we discussed an idea that I think is worth trying at least for icehouse-1 to see if we like it or not. The idea is that *every* blueprint starts out at a Low priority, which means best effort, but no promises. For a blueprint to get prioritized higher, it should have 2 nova-core members signed up to review the resulting code. Huge +1 to this. I'm in favor of the whole plan, but specifically the prioritization piece is very important, IMHO. I too am in favor of the idea. It is just not clear how 2 Nova cores will be signed up. Good point, there was no detail on that. I propose just comments on the blueprint whiteboard. It can be something simple like this to indicate that Dan and I have agreed to review the code for something: nova-core reviewers: russellb, dansmith -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
