Re: [Openstack] Feature Freeze status
On Thu, 31 Mar 2011 10:02:34 +0200 Soren Hansen so...@openstack.org wrote: 2011/3/31 FUJITA Tomonori fujita.tomon...@lab.ntt.co.jp: Sounds like the importance of blueprints is overrated. If you look at Linux kernel development (more than ten times developers and development speed, I guess), The Linux kernel development process uses a number of other approaches to deal with its volume. There a number of different staging trees. There are topic-based ones, like the one for people working on 'topic-based' is one of ways to manage a git tree so it's unrelated with this top but as you said, the patch acceptance load is distributed across a number of lieutenants. If there are more than 1,000 developers, you need to do that, I think. I expect OpenStack will need to do the similar with developers increased. You don't? you see that lots of developers can work well without something like blueprints. Just so we're clear: Linux has merge windows as well. New features simply do not get to go into mainline outside of these merge windows. Yeah, but you can propose new features (patches) any time. It's not uncommon for changes to take several kernel releases from being proposed until they're actually accepted into the tree. Also, noone promises that your stuff will get reviewed. Ever. We do. If you propose stuff in time, we promise it will at least be reviewed once before feature freeze. Hmm, that 'promise' works (and will work in the future)? I already saw the shortage of reviewing (and I even saw that a half-baked feature without enough reviewed was merged and reverted). I bet that we'll face more serious shortage of reviewing with developers increasing. In genera, developers prefer to write own code rather than reviewing. We can't change the nature. I think that there is no such 'promise' for reviewing in a large OSS. If your code can't get enough reviewers, your code might be not useful enough for the project. It's sorta evolutionary theory. The better code has the better chance to be merged quickly. The features are prioritized fairly. I don't think that openness and transparency are not related with blueprints. You can simply discuss, make a decision, etc on public mailing lists. That's merely transposing the exact same process, isn't it? If the goal is to have features described, whether you have to type it in one place or another shouldn't matter much. What does matter, though, is tracking. I think it's common consensus that mailing lists make *dreadful* bug trackers. I'm not sure I understand why it would be much better for tracking blueprints or whatever you'd call them in that context? Oops, I didn't mean that. I just meant that you can achieve openness and transparency without blueprints. I read some arguments in this thread, something like blueprints is a must for openness and transparency. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Feature Freeze status
On Fri, Apr 1, 2011 at 8:41 AM, FUJITA Tomonori fujita.tomon...@lab.ntt.co.jp wrote: I bet that we'll face more serious shortage of reviewing with developers increasing. In genera, developers prefer to write own code rather than reviewing. We can't change the nature. Yes, this is certainly true. I'm hoping that the PTL(s) will help give some stability to the current process. I think that there is no such 'promise' for reviewing in a large OSS. If your code can't get enough reviewers, your code might be not useful enough for the project. It's sorta evolutionary theory. The better code has the better chance to be merged quickly. The features are prioritized fairly. Hmm, I disagree. What is actually more likely is that the clique of developers that the proposing author is friendly/familiar will review their merge proposal before other ones. It's not really about good code versus bad code. Again, it's about prioritizing the reviews for folks that follow the procedures the community has agreed on. If the community wants to change the procedures that it has agreed on, it should do so at the next summit. I *think* that's what Vishy was saying and that I agree with. -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Feature Freeze status
On Fri, Apr 1, 2011 at 9:19 AM, FUJITA Tomonori fujita.tomon...@lab.ntt.co.jp wrote: Yeah, but discussion on the mailing list before the summit is useful (developers who can't attend the summit are able to discuss, I might not be able to make it too). True enough :) Didn't mean to suggest the discussion on the ML wasn't useful. It certainly has been! Cheers, jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Feature Freeze status
2011/4/1 FUJITA Tomonori fujita.tomon...@lab.ntt.co.jp: On Thu, 31 Mar 2011 10:02:34 +0200 Soren Hansen so...@openstack.org wrote: 2011/3/31 FUJITA Tomonori fujita.tomon...@lab.ntt.co.jp: Sounds like the importance of blueprints is overrated. If you look at Linux kernel development (more than ten times developers and development speed, I guess), The Linux kernel development process uses a number of other approaches to deal with its volume. There a number of different staging trees. There are topic-based ones, like the one for people working on 'topic-based' is one of ways to manage a git tree so it's unrelated with this top Um, no. I realise git has a concept of topic branches, but this is not what I'm referring to at all. David Miller, for instance, maintains git.kernel.org/.../davem/net-2.6.git which is where all the interesting networking related stuff goes on. Every once in a while, this gets merged into Linus' linux-2.6 tree. I expect OpenStack will need to do the similar with developers increased. You don't? Quite probably. When the time comes. Different scales of development community warrant different approaches. you see that lots of developers can work well without something like blueprints. Just so we're clear: Linux has merge windows as well. New features simply do not get to go into mainline outside of these merge windows. Yeah, but you can propose new features (patches) any time. Sure. You can do that in OpenStack, too. People just still expect to get stuff discussed/reviewed/merged outside of this window. That's what we're talking about. I don't believe our core development team is large enough to support splitting our efforts like this. When we're past feature freeze, we all need to focus on QA'ing what we've got already. There are plenty of bugs to fix. It's not uncommon for changes to take several kernel releases from being proposed until they're actually accepted into the tree. Also, noone promises that your stuff will get reviewed. Ever. We do. If you propose stuff in time, we promise it will at least be reviewed once before feature freeze. Hmm, that 'promise' works (and will work in the future)? If we want it to, it can. If noone does reviews and just everyone keeps working on their own snazzy, new features, it won't. That's really the core of this problem. I already saw the shortage of reviewing (and I even saw that a half-baked feature without enough reviewed was merged and reverted). Yes! EXACTLY! Because people who ought to be reviewing aren't. I bet that we'll face more serious shortage of reviewing with developers increasing. In genera, developers prefer to write own code rather than reviewing. We can't change the nature. Then people ought to grow up. Seriously. They should grow up and stop saying that they're going to review stuff, or they should start actually reviewing stuff. I think that there is no such 'promise' for reviewing in a large OSS. If your code can't get enough reviewers, your code might be not useful enough for the project. It's sorta evolutionary theory. The better code has the better chance to be merged quickly. The features are prioritized fairly. I think we have quite different definitions of fair. We're on a mission: to produce the ubiquitous Open Source Cloud Computing platform that will meet the needs of public and private clouds regardless of size, by being simple to implement and massively scalable. We can't pretend to meet everyone's needs if we only accept patches that are fun to review or that we wrote ourselves. -- Soren Hansen Ubuntu Developer http://www.ubuntu.com/ OpenStack Developer http://www.openstack.org/ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Feature Freeze status
On Fri, 1 Apr 2011 22:02:02 +0200 Soren Hansen so...@openstack.org wrote: I already saw the shortage of reviewing (and I even saw that a half-baked feature without enough reviewed was merged and reverted). Yes! EXACTLY! Because people who ought to be reviewing aren't. I think that we need to admint that there isn't enough reviewing power for that 'promise'. Instead merge only patches that are reviewed well. If people get frustrated with the speed of merging patches, some developers start to review patches. I bet that we'll face more serious shortage of reviewing with developers increasing. In genera, developers prefer to write own code rather than reviewing. We can't change the nature. Then people ought to grow up. Seriously. They should grow up and stop saying that they're going to review stuff, or they should start actually reviewing stuff. I don't think that people grow up in the way that you expect. I feel that I listen to the shortage of reviewing discussion at every kernel summit. Even though there are more than 1,000 kernel developers. I think that there is no such 'promise' for reviewing in a large OSS. If your code can't get enough reviewers, your code might be not useful enough for the project. It's sorta evolutionary theory. The better code has the better chance to be merged quickly. The features are prioritized fairly. I think we have quite different definitions of fair. We're on a mission: to produce the ubiquitous Open Source Cloud Computing platform that will meet the needs of public and private clouds regardless of size, by being simple to implement and massively scalable. We can't pretend to meet everyone's needs if we only accept patches that are fun to review or that we wrote ourselves. The majority of OpenStack developers are paid for OpenStack development, right? I don't think that we need to worry about such. Look at Linux kernel development. The mojority are professionals and understand what they need to do. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Feature Freeze status
I'm gonna +1 Todd. Actually, apache server has a great dev process. They have goals for releases, but people are welcome to submit patches to their mailing list any time, get comments on them, then they're merged if and when people vote them as ready. -- Mike ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Feature Freeze status
On Thu, Mar 31, 2011 at 5:21 PM, Michael Barton mike-launch...@weirdlooking.com wrote: I'm gonna +1 Todd. Actually, apache server has a great dev process. They have goals for releases, but people are welcome to submit patches to their mailing list any time, get comments on them, then they're merged if and when people vote them as ready. We're talking about prioritizing the merge proposals that pile up based on whether such public discussion has occurred on the ML and/or has been documented on a blueprint. We're not talking about limiting people's ability to submit patches. We're talking about prioritizing the reviews that have been proposed... -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Feature Freeze status
There has been a lot of great discussion and airing-out around blueprint and merge process. I think some good points have been raised on all sides. I'd like to weigh in with some perspective-changing suggestions about how we can manage specs and blueprints that I think everyone will be happy with. Basically we are all in the process of learning how to manage a large open-source project with 100+ developers, and I think managing a group that large by throwing everyone into a pool and hoping that all of the developers can collaborate effectively is a bit optimistic. The suggestions that I outline below are not radical changes from how we are currently doing things, but hopefully it will help clarify the process. Blueprints People have extolled the value of blueprints many times, and I agree that they are very valuable. But I think blueprints are much more valuable from a project management perspective than they are from an 'in-the-weeds' coding perspective. I would suggest that blueprints are used to give a broad overview of an intended feature and enough technical information for the PTL and other teams to ensure that work isn't being duplicated. Since we all work in teams, I think it is reasonable to expect every team to have a contact person that is responsible for ensuring that the team's blueprints are up-to-date with what they are actually working on. Internal to the team this can be managed however they see fit. It can be offloaded to individual developers or handled by a project manager, etc. If we can all strive to follow this limited use of blueprints, I think it gives us the advantages that they provide for project management without putting too much strain on the developers. Specs Detailed specs beyond a brief technical overview should not be required in all cases. It is strongly recommended (but not required) for a team to make available any internal specifications that they are using. For small features, a simple link to a public branch is enough. Detailed Specs should be required in the following cases: * A large feature that touches a lot of the code * Code that will need multiple merge proposals * Features that are being worked on by multiple teams * A feature that is blocking features by other teams. I think we could Branches Teams should be encouraged to keep their branches in the public as work goes on. This allows curious community members to drill down into the current development and see what is going on. This is especially important for teams using agile development. Merge Merges should be evaluated on merit. If we get a large feature without an associated blueprint/spec, we can help educate the developer on the blueprint / spec process, but i don't think we should block merging if the feature is well designed and tested. Obviously if the feature interferes with other blueprints in the pipeline, we can block it. In conclusion, I strongly agree with soren's comment that the core developers should be following the suggested process, and I will mea culpa in my own avoidance of blueprints. I think a lot of the issues the developers have had are due to a feeling that it is a) complicated and b) not valuable. Hopefully with the understanding of the value that has been provided in this thread and the clarification and suggestions I've provided, we can all improve our teamwork. Please let me know if I've missed anything. Vish___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Feature Freeze status
Well said. I don't disagree with any of your points or suggestions below. Looking forward to a productive chat about it at the summit in a few weeks. -jay On Thu, Mar 31, 2011 at 6:09 PM, Vishvananda Ishaya vishvana...@gmail.com wrote: There has been a lot of great discussion and airing-out around blueprint and merge process. I think some good points have been raised on all sides. I'd like to weigh in with some perspective-changing suggestions about how we can manage specs and blueprints that I think everyone will be happy with. Basically we are all in the process of learning how to manage a large open-source project with 100+ developers, and I think managing a group that large by throwing everyone into a pool and hoping that all of the developers can collaborate effectively is a bit optimistic. The suggestions that I outline below are not radical changes from how we are currently doing things, but hopefully it will help clarify the process. Blueprints People have extolled the value of blueprints many times, and I agree that they are very valuable. But I think blueprints are much more valuable from a project management perspective than they are from an 'in-the-weeds' coding perspective. I would suggest that blueprints are used to give a broad overview of an intended feature and enough technical information for the PTL and other teams to ensure that work isn't being duplicated. Since we all work in teams, I think it is reasonable to expect every team to have a contact person that is responsible for ensuring that the team's blueprints are up-to-date with what they are actually working on. Internal to the team this can be managed however they see fit. It can be offloaded to individual developers or handled by a project manager, etc. If we can all strive to follow this limited use of blueprints, I think it gives us the advantages that they provide for project management without putting too much strain on the developers. Specs Detailed specs beyond a brief technical overview should not be required in all cases. It is strongly recommended (but not required) for a team to make available any internal specifications that they are using. For small features, a simple link to a public branch is enough. Detailed Specs should be required in the following cases: * A large feature that touches a lot of the code * Code that will need multiple merge proposals * Features that are being worked on by multiple teams * A feature that is blocking features by other teams. I think we could Branches Teams should be encouraged to keep their branches in the public as work goes on. This allows curious community members to drill down into the current development and see what is going on. This is especially important for teams using agile development. Merge Merges should be evaluated on merit. If we get a large feature without an associated blueprint/spec, we can help educate the developer on the blueprint / spec process, but i don't think we should block merging if the feature is well designed and tested. Obviously if the feature interferes with other blueprints in the pipeline, we can block it. In conclusion, I strongly agree with soren's comment that the core developers should be following the suggested process, and I will mea culpa in my own avoidance of blueprints. I think a lot of the issues the developers have had are due to a feeling that it is a) complicated and b) not valuable. Hopefully with the understanding of the value that has been provided in this thread and the clarification and suggestions I've provided, we can all improve our teamwork. Please let me know if I've missed anything. Vish ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Feature Freeze status
From: Todd Willey [t...@ansolabs.com] I, too, appreciate people taking their time to write blueprints. I also appreciate people who take time to write code, even if it doesn't come with a blueprint. People who take their creative energy to contribute to something I love earn enough favor that my default state is to try and accept their contribution without imposing additional barriers to entry. Maybe a patch comes in that it isn't reasonable to merge, but we never know unless we actually look at it with the mindset of trying to accept it. I still expect that people give blueprint-level details in merge proposals so that we can follow the code with an understanding of what you're trying to accomplish, otherwise we'd do right to mark it Needs Information and get the clarity we need. Todd brings up an interesting point here. I think of a blueprint as requirements gathering vs design. Code without agreed-upon requirements has the risk of solving the wrong problems. It doesn't have to be a tome ... notes are fine. The example I can cite is directapi/authn/authz. While having the code is a great discussion point, the spec and notes don't really give enough to explain: * the use cases (happy day and exceptions) * the actors involved * the cons (lots of pro's given) Having a blueprint to go with it would have given us time to think about the problem while the code was being put together. Now, I have a body of code to look at, but now I need to understand the gotcha's of adopting it. The How's and the Why's. A blueprint would make this much easier. Secondly, while I'm a strong advocate of code over talk I think there is an unwritten assumption that, simply because there is code, it deserves merge into the code stream. If code takes the place of a blueprint, it should get the same weight as a blueprint. That is, it may get rejected outright or alternative designs recommended to the point the code is a throwaway. $0.02 -S PS And don't get me wrong, directapi/authn/z are all impressive branches, completely worthy of our full attention. Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Feature Freeze status
Ewan Mellor wrote: Blueprints are not about managing expectations. Blueprints are about telling other people what you're working on. This is important, because I don't think that anyone is in a position to judge whether they are working on a self-contained change or whether they're not going to interfere _unless_ they are telling people what they are working on. +1 The cost of filing a blueprint (as soon as you start thinking about a feature) is really small compared to the advantages stated above. I guess I'm not seeing any advantage in *not filing* a blueprint. I hope it's not about saving 5 minutes. I hope it's not about avoiding public discussion that developing in the open can trigger... -- Thierry Carrez (ttx) Release Manager, OpenStack ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Feature Freeze status
Todd Willey wrote: I read the initial problem as some things don't have blueprints and are developed in isolation which I didn't see as a problem. Now i see it as: * coherency of vision * review priority This should partially address the review priority issue: http://wiki.openstack.org/reviewslist/ Note that it relies on bug and blueprint links to actually evaluate that priority. * branches dropping right before freeze (I don't think this is solvable) I'd rather solve it by education rather than repression, but it will probably take a few bad experiences (like rejecting late branches) for people to realize that it's counter-productive. -- Thierry Carrez (ttx) Release Manager, OpenStack ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Feature Freeze status
-Original Message- From: Todd Willey [mailto:t...@ansolabs.com] Sent: 29 March 2011 04:08 To: Ewan Mellor Cc: Jay Pipes; openstack@lists.launchpad.net Subject: Re: [Openstack] Feature Freeze status On Mon, Mar 28, 2011 at 7:42 PM, Ewan Mellor ewan.mel...@eu.citrix.com wrote: -Original Message- From: Todd Willey Sent: 28 March 2011 21:11 To: Jay Pipes Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] Feature Freeze status [Snip] Clearly blueprints are about process and not about code. Merge proposals are a hybrid of code and process. Blueprints are about managing expectations, whereas merge proposals are about managing code. I think if you are working on a self-contained change you don't need to manage the expectations of people working on other parts of the system because you're not going to interfere, and their expectations about how they should write code can go unchanged. You do, however, need to have a process required to get your changes accepted into the code, and need to outline your reasoning and implementation goals. I think that this paragraph is a great exposition of why I (and others) disagree with you. Blueprints are not about managing expectations. Blueprints are about telling other people what you're working on. This is important, because I don't think that anyone is in a position to judge whether they are working on a self-contained change or whether they're not going to interfere _unless_ they are telling people what they are working on. Blueprints are great for the reason you and Rick have stated. They let all types of people who are interested in monitoring the development and planning their own development and implementation plan more effectively. I think of unplanned features as an extra gift on top of all of this that we should accept gratefully. I'm not saying we can know before propose-time if a feature is isolated. But, if at the end it turns out it is in fact isolated, then I see no reason we shouldn't welcome it with a minimum of drama. I don't agree. Unplanned features aren't an extra gift -- they're extra work for everyone. Thierry is trying to stabilize a release and get it out on time. He can't do that if people come along saying here's an extra gift that you now need to give time to review. We're already short on resources, and Thierry needs to prioritize based on the capacity that we collectively have. For example, Citrix has bumped a number of features to Diablo, because it was clear that we wouldn't get everything reviewed and merged and stabilized in time unless we deferred low priority work. Unplanned features just put all that in jeopardy. If someone comes along with a random branch as a gift, then it's not unreasonable for us to say Thank you for that, we'll take it in the next unstable period in a couple of months. In the meantime, please write a brief blueprint so that we don't forget about it. A blueprint doesn't have to be burdensome. Big features need a long time at the planning stage, but for other things there's no reason why the blueprint should take more than half-an-hour. I don't think it's unreasonable to ask anybody to spend half-an-hour writing down what their new shiny thing is, how it works, and how to test it. Cheers, Ewan. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Feature Freeze status
Hi Thierry, in addition to the technical sessions for Nova, Swift, and Glance at the design summit, I have asked Stephen to add Project and Release management. You will own the time slots, once the PTL's are on board we will discuss how best to schedule the sessions. Thanks, John -Original Message- From: openstack-bounces+john=openstack@lists.launchpad.net [mailto:openstack-bounces+john=openstack@lists.launchpad.net] On Behalf Of Thierry Carrez Sent: Monday, March 28, 2011 4:48 AM To: openstack@lists.launchpad.net Subject: [Openstack] Feature Freeze status Hi everyone, We have had quite a few branches that filed for feature freeze exceptions, and the following have been granted an exception for late merging until Tuesday EOD, so they need your attention: https://code.launchpad.net/~blamar/nova/openstack-api-1-1-images/+merge/5394 2 https://code.launchpad.net/~citrix-openstack/nova/xenapi-vlan-network-manage r/+merge/53660 https://code.launchpad.net/~zulcss/nova/nova-lxc/+merge/51469 https://code.launchpad.net/~justin-fathomdb/nova/volumes-api/+merge/54464 https://code.launchpad.net/~termie/nova/libvirt_snapshot/+merge/54621 https://code.launchpad.net/~sleepsonthefloor/nova/vnc_console/+merge/54805 As a sidenote, some of those branches are cool and self-contained features that are relatively-safe for the release. So they were accepted as far as the FFe is concerned, despite not having blueprints and being late. I'd like to say that I dislike this type of behind closed doors development, where things are developed without a blueprint, bug or linked branch and then thrown very late into the review queue. We do an open source project and we promised open development. That includes doing development in the open, and that's why we use blueprints to track stuff under development, so that anyone can look, learn about it and comment. Looking at what happened in cactus I can see that some people are not happy with these idea[l]s... Should we discuss this at some point during the design summit ? -- Thierry Carrez (ttx) Release Manager, OpenStack ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Feature Freeze status
On Mon, Mar 28, 2011 at 5:48 AM, Thierry Carrez thie...@openstack.org wrote: I'd like to say that I dislike this type of behind closed doors development, where things are developed without a blueprint, bug or linked branch and then thrown very late into the review queue. We do an open source project and we promised open development. That includes doing development in the open, and that's why we use blueprints to track stuff under development, so that anyone can look, learn about it and comment. Looking at what happened in cactus I can see that some people are not happy with these idea[l]s... Should we discuss this at some point during the design summit ? +10. -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Feature Freeze status
No objection to a discussion during the summit, but I've been able to watch all of these branches and others evolve here: https://code.launchpad.net/nova For example, when I wanted to add VNC support because my system wouldn't boot, it was easy for me to look at the vnc_console branch and get the libvirt XML magic from there. If I was feeling brave, I could have grabbed the whole branch and merged it into my tree. That feels _very_ open to me. jaypipes_troll Just another benefit of bazaar being a centralized version control system, I guess /jaypipes_troll Justin ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Feature Freeze status
On Mon, Mar 28, 2011 at 1:38 PM, Justin Santa Barbara jus...@fathomdb.com wrote: No objection to a discussion during the summit, but I've been able to watch all of these branches and others evolve here: https://code.launchpad.net/nova For example, when I wanted to add VNC support because my system wouldn't boot, it was easy for me to look at the vnc_console branch and get the libvirt XML magic from there. If I was feeling brave, I could have grabbed the whole branch and merged it into my tree. That feels _very_ open to me. Rubbish. Open development means knowing the general directions and specifications that people are working on by open discussions, open blueprints/specs, and active communication between teams. I can go to github and see how many people forked this (ugh.). That doesn't give me any clue as to what people are attempting to do with the code in the long term. The problem we've been having revolves around the fact that we have dozens of developers working on Nova, with no real guidance as to the long-term direction of the project, and teams just doing whatever the heck they want to, without ML discussion and blueprints, and then expecting people to just merge branches a few days before feature freeze. jaypipes_troll Just another benefit of bazaar being a centralized version control system, I guess /jaypipes_troll I don't know what the heck you are talking about. -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Feature Freeze status
The problem we've been having revolves around the fact that we have dozens of developers working on Nova, with no real guidance as to the long-term direction of the project, and teams just doing whatever the heck they want to, without ML discussion and blueprints, and then expecting people to just merge branches a few days before feature freeze. I have high hopes that the newly elected PTL’s will provide the guidance and leadership to get more effective in developing OpenStack projects. John -Original Message- From: openstack-bounces+john=openstack@lists.launchpad.net [mailto:openstack-bounces+john=openstack@lists.launchpad.net] On Behalf Of Jay Pipes Sent: Monday, March 28, 2011 12:54 PM To: Justin Santa Barbara Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] Feature Freeze status On Mon, Mar 28, 2011 at 1:38 PM, Justin Santa Barbara jus...@fathomdb.com wrote: No objection to a discussion during the summit, but I've been able to watch all of these branches and others evolve here: https://code.launchpad.net/nova For example, when I wanted to add VNC support because my system wouldn't boot, it was easy for me to look at the vnc_console branch and get the libvirt XML magic from there. If I was feeling brave, I could have grabbed the whole branch and merged it into my tree. That feels _very_ open to me. Rubbish. Open development means knowing the general directions and specifications that people are working on by open discussions, open blueprints/specs, and active communication between teams. I can go to github and see how many people forked this (ugh.). That doesn't give me any clue as to what people are attempting to do with the code in the long term. The problem we've been having revolves around the fact that we have dozens of developers working on Nova, with no real guidance as to the long-term direction of the project, and teams just doing whatever the heck they want to, without ML discussion and blueprints, and then expecting people to just merge branches a few days before feature freeze. jaypipes_troll Just another benefit of bazaar being a centralized version control system, I guess /jaypipes_troll I don't know what the heck you are talking about. -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Feature Freeze status
Rubbish. Open development means knowing the general directions and specifications that people are working on by open discussions, open blueprints/specs, and active communication between teams. I can go to github and see how many people forked this (ugh.). That doesn't give me any clue as to what people are attempting to do with the code in the long term. I am not the biggest fan of LP, but the 'recent commits' feature on LP is awesome. If I want to know what people are really working on, I just look at the most recent commits: Cerberus is working on something cool involving notifications, presumably the first step in addressing the issue that we're almost entirely request-driven The titan team is fixing the date/time parsing bug with glance images The titan team is working on support for changing passwords ZulCss is working on LXC support The USC team is working on HPC stuff on hardware that's unfairly cool Termie's working on libvirt-snapshot support Termie's clarifying the docstring rules etc. That gives me a much better feel for the pulse of the project than anything else - the blueprints feel like reading a printed newspaper - full of yesterday's news. It doesn't answer why, but we should probably make a bigger push on the blueprints to make sure they always answer the why question. Justin ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Feature Freeze status
Open == Accessible. Open != Verbose. I'm willing to discuss more at the design summit, but my biggest concern is that we let the most people possible can contribute. This includes those who work behind closed doors on their own pet projects. This thread started specifically in response to a few branches that didn't hinder anyone else's work, but also didn't have blueprints. So what? Blueprints should only be required for coordination when doing large work that can conflict with others'. Code should be the only artifact required to contribute, as that will keep contribution more accessible. If these proposers had run afoul of any other work in progress where the other branch _did_ have a blueprint, we could have just said Sorry, you'll have to rebase after this other thing goes in, and coordinate better in the future, at which point they could have kept working to get their patch to land or walk away without getting it approved. Reviews are open so people working on branches that have blueprints can complain if a rogue branch gets proposed that would interfere with their plans. I really feel like we're making a problem where none exists. You can certainly craft a scenario where a lack of coordination causes a problem in a branch like this, but we don't have actual evidence it will happen. If it does, we can just push back against the proposer to fix things. Whats the problem with that? -todd[1] On Mon, Mar 28, 2011 at 1:53 PM, Jay Pipes jaypi...@gmail.com wrote: On Mon, Mar 28, 2011 at 1:38 PM, Justin Santa Barbara jus...@fathomdb.com wrote: No objection to a discussion during the summit, but I've been able to watch all of these branches and others evolve here: https://code.launchpad.net/nova For example, when I wanted to add VNC support because my system wouldn't boot, it was easy for me to look at the vnc_console branch and get the libvirt XML magic from there. If I was feeling brave, I could have grabbed the whole branch and merged it into my tree. That feels _very_ open to me. Rubbish. Open development means knowing the general directions and specifications that people are working on by open discussions, open blueprints/specs, and active communication between teams. I can go to github and see how many people forked this (ugh.). That doesn't give me any clue as to what people are attempting to do with the code in the long term. The problem we've been having revolves around the fact that we have dozens of developers working on Nova, with no real guidance as to the long-term direction of the project, and teams just doing whatever the heck they want to, without ML discussion and blueprints, and then expecting people to just merge branches a few days before feature freeze. jaypipes_troll Just another benefit of bazaar being a centralized version control system, I guess /jaypipes_troll I don't know what the heck you are talking about. -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Feature Freeze status
On Mon, Mar 28, 2011 at 2:32 PM, Todd Willey t...@ansolabs.com wrote: Open == Accessible. Open != Verbose. I'm willing to discuss more at the design summit, but my biggest concern is that we let the most people possible can contribute. This includes those who work behind closed doors on their own pet projects. It's not about letting people contribute. It's an open source project. Anyone can contribute. Anyone can work on a fork of their PetProjectStack. But when it comes to merge proposal time, and especially the backlog that always shows up around Feature Freeze, I couldn't care less about PetProjectStack unless it has a blueprint or bug tied to it that lets me know the patch fixes something or adds something to the project that has been debated and discussed. This thread started specifically in response to a few branches that didn't hinder anyone else's work, but also didn't have blueprints. Not true. There were only a few branches that didn't have bugs or blueprints assigned to them. Those branches are what Thierry and myself were talking about. So what? Blueprints should only be required for coordination when doing large work that can conflict with others'. Code should be the only artifact required to contribute, as that will keep contribution more accessible. IMHO, this doesn't work for projects with dozens of developers and many teams all working towards a unified goal (you know, that whole this release is about X thing?). It works spectacularly well for projects with developers all working on their own forks of stuff that want to do their own hacks but don't care about the direction of the project as a whole. In other words, the blueprint/discussion process is designed to bring unity to the project's direction and goals. it's about *process*, not code. If these proposers had run afoul of any other work in progress where the other branch _did_ have a blueprint, we could have just said Sorry, you'll have to rebase after this other thing goes in, and coordinate better in the future, at which point they could have kept working to get their patch to land or walk away without getting it approved. Reviews are open so people working on branches that have blueprints can complain if a rogue branch gets proposed that would interfere with their plans. We complain because rogue branches interfere with the process of trying to figure out what branches are to be reviewed in which order and priority. They add to the chaos when I can't answer the why question that Justin alluded to earlier. I really feel like we're making a problem where none exists. You can certainly craft a scenario where a lack of coordination causes a problem in a branch like this, but we don't have actual evidence it will happen. If it does, we can just push back against the proposer to fix things. Whats the problem with that? If the problem didn't exist, I wouldn't bring it up. -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Feature Freeze status
2011/3/28 Thierry Carrez thie...@openstack.org: I'd like to say that I dislike this type of behind closed doors development, where things are developed without a blueprint, bug or linked branch and then thrown very late into the review queue. I'd appreciate a discussion about this at the summit, too. Just to register my opinion ahead of time: Were it from outside people (i.e. not one of our regulars), I'd have less of a problem with this. It's hard to keep a straight face while telling people that they need to follow this or that process when we can't even manage to do so ourselves. We should be leading by example. -- Soren Hansen Ubuntu Developer http://www.ubuntu.com/ OpenStack Developer http://www.openstack.org/ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Feature Freeze status
Blueprints serve three purposes. I don't claim they do them well, or that we are using them well 1) they help us schedule technical discussions at the summit. We could obviously do it some other way, but that is on of the current uses. 2) They let the various dev groups know what is being worked on, so we don't duplicate efforts. and lets multiple groups know the approved architecture based on the summit discussions. 3) They let non-technical people follow the development cycle. This is the most important use, in my opinion. There are project managers, product managers, marketing and PR people, and executives, and more. They all need to either follow the dev cycle, or know what features are going to hit, or miss, ahead of release. It is not reasonable for those folks to have to read the MP's TBH, right now, blueprints are not optimal. The Launchpad team is rewriting blueprints to use the Launchpad bug engine. This hopefully will fix some of the glaring problems. Like not being able to have a linear discussion. Rick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Feature Freeze status
-Original Message- From: Todd Willey Sent: 28 March 2011 21:11 To: Jay Pipes Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] Feature Freeze status [Snip] Clearly blueprints are about process and not about code. Merge proposals are a hybrid of code and process. Blueprints are about managing expectations, whereas merge proposals are about managing code. I think if you are working on a self-contained change you don't need to manage the expectations of people working on other parts of the system because you're not going to interfere, and their expectations about how they should write code can go unchanged. You do, however, need to have a process required to get your changes accepted into the code, and need to outline your reasoning and implementation goals. I think that this paragraph is a great exposition of why I (and others) disagree with you. Blueprints are not about managing expectations. Blueprints are about telling other people what you're working on. This is important, because I don't think that anyone is in a position to judge whether they are working on a self-contained change or whether they're not going to interfere _unless_ they are telling people what they are working on. For example, there have been many occasions when we (Citrix) have said oh, we don't need to work on that, because Rackspace have it covered or we'd best mention this to Rackspace, because they're working on something else that is _bound_ to conflict. There have also been times when we've said it sure would have been great to know about that in advance, and save ourselves half-a-dozen trunk merges. Blueprints are important because they save other people wasting their time. That's either because they get to avoid duplicating your effort, or because they want to work in the same code and get to schedule around you so that the merge conflicts aren't insane. Maybe, if you're really lucky, they might actually have some good ideas and can make suggestions to you. Blueprints should be regarded in the same way as docstrings. Sure, you can write the code faster without docstrings, and the code will work just as well, but it will take everyone else twice as long to figure out what it was you are trying to do. Blueprints are the same, just at a different phase in the process. You can definitely write the code without writing the blueprint first, but everyone else would sure appreciate it if you put half-an-hour into the blueprint. That way, we'll know what it is you're trying to do. Thanks, Ewan. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Feature Freeze status
+1 From: Ewan Mellor [ewan.mel...@eu.citrix.com] Blueprints should be regarded in the same way as docstrings. Sure, you can write the code faster without docstrings, and the code will work just as well, but it will take everyone else twice as long to figure out what it was you are trying to do. Blueprints are the same, just at a different phase in the process. You can definitely write the code without writing the blueprint first, but everyone else would sure appreciate it if you put half-an-hour into the blueprint. That way, we'll know what it is you're trying to do. Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Feature Freeze status
On Mon, Mar 28, 2011 at 7:42 PM, Ewan Mellor ewan.mel...@eu.citrix.com wrote: -Original Message- From: Todd Willey Sent: 28 March 2011 21:11 To: Jay Pipes Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] Feature Freeze status [Snip] Clearly blueprints are about process and not about code. Merge proposals are a hybrid of code and process. Blueprints are about managing expectations, whereas merge proposals are about managing code. I think if you are working on a self-contained change you don't need to manage the expectations of people working on other parts of the system because you're not going to interfere, and their expectations about how they should write code can go unchanged. You do, however, need to have a process required to get your changes accepted into the code, and need to outline your reasoning and implementation goals. I think that this paragraph is a great exposition of why I (and others) disagree with you. Blueprints are not about managing expectations. Blueprints are about telling other people what you're working on. This is important, because I don't think that anyone is in a position to judge whether they are working on a self-contained change or whether they're not going to interfere _unless_ they are telling people what they are working on. Blueprints are great for the reason you and Rick have stated. They let all types of people who are interested in monitoring the development and planning their own development and implementation plan more effectively. I think of unplanned features as an extra gift on top of all of this that we should accept gratefully. I'm not saying we can know before propose-time if a feature is isolated. But, if at the end it turns out it is in fact isolated, then I see no reason we shouldn't welcome it with a minimum of drama. For example, there have been many occasions when we (Citrix) have said oh, we don't need to work on that, because Rackspace have it covered or we'd best mention this to Rackspace, because they're working on something else that is _bound_ to conflict. There have also been times when we've said it sure would have been great to know about that in advance, and save ourselves half-a-dozen trunk merges. Blueprints are important because they save other people wasting their time. That's either because they get to avoid duplicating your effort, or because they want to work in the same code and get to schedule around you so that the merge conflicts aren't insane. Maybe, if you're really lucky, they might actually have some good ideas and can make suggestions to you. I understand that working without blueprints is more often the more painful way to do things. I'm not saying it's good, I'm only saying that there are cases where it is _acceptable_. Breaking things, making major design decisions without discussion, and ignoring our platform goals should get things rejected. Always. We can also still offer feedback and suggestions in a MP, so I think the window of time that people have to be really lucky is non-zero even in the absence of a BP. With core review days and targeting specific reviewers by name we should continue to move this out of the realm of luck and into a process designed to get the right people providing that meaningful feedback. Blueprints should be regarded in the same way as docstrings. Sure, you can write the code faster without docstrings, and the code will work just as well, but it will take everyone else twice as long to figure out what it was you are trying to do. Blueprints are the same, just at a different phase in the process. You can definitely write the code without writing the blueprint first, but everyone else would sure appreciate it if you put half-an-hour into the blueprint. That way, we'll know what it is you're trying to do. I, too, appreciate people taking their time to write blueprints. I also appreciate people who take time to write code, even if it doesn't come with a blueprint. People who take their creative energy to contribute to something I love earn enough favor that my default state is to try and accept their contribution without imposing additional barriers to entry. Maybe a patch comes in that it isn't reasonable to merge, but we never know unless we actually look at it with the mindset of trying to accept it. I still expect that people give blueprint-level details in merge proposals so that we can follow the code with an understanding of what you're trying to accomplish, otherwise we'd do right to mark it Needs Information and get the clarity we need. Thanks, Ewan. I get that blueprints are useful, all I'm advocating is that we don't beat people over the head with it. -todd[1] ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https
