Development process (was: working together)
Thanks Sandeep. Would we all be comfortable adopting this process going forward, hopefully reducing friction, bugs and problems in general? I assume +1 from me and Sandeep so far. /Johan Sandeep Tata wrote: Johan, the wiki pages are great! I think they will help iron out our process for contributing and committing. (I added a pointer to the formatting conventions in HowToContribute , can't think of anything else to add) http://cwiki.apache.org/confluence/display/CSDR/HowToContribute http://cwiki.apache.org/confluence/display/CSDR/HowToCommit http://cwiki.apache.org/confluence/display/CSDR/HowToRelease A short summary and description of why these points make sense: * Patch-only evolution of code, attached to a jira issue * At least one +1 on each issue before it can be committed, -1 stops the patch. Those two points would make sure that if someone disagrees with a change, a refactoring etc, they have a chance to voice their opinion and steer it into the right direction. * Trunk is not considered stable, but must pass unit tests * Any non trivial change should include unit tests * When a branch is created to prepare for a release extra effort is put into QA to make sure the release is as stable as possible. Point releases would then go out to fix issues found after the release was done. * Once a release has been out for a while and people are using it in production without problems it is upgraded to stable status. The purpose of these points is to encourage a vibrant codebase, to not be afraid of for example refactoring if it improves the code readability or testability. I appreciate that Cassandra is a complex system and that changes might have unwanted side effects, but hopefully adding tests and code reviews will reduce those. As a final catch-all the release candidate and stable release process should help end users avoid bugs. Thoughts on the wiki pages? Do they help resolve some of the problems? /Johan Sandeep Tata wrote: Thoughts inline: So the problems I am seeing are: 1. We elected a committer without real community consensus. The barrier of entry was unnatural low on this one. On the other hand we need non-FB committers for the graduation. The more the better. (No reason for low entry barrier though!) I think everyone (including the FB guys) agree that Jonathan has been working hard to help move the codebase forward. He has been quick to revert changes that broke the code that the FB guys had in the pipeline and have committed since. I think much of the friction comes from not having a process, which takes us to Torsten's #2: 2. A missing definition of development process: - What is considered a valid code review? - How much are changes discussed up-front? - What is the roadmap? ...for whom? (weighted as a community) This is probably where we need most work. Here are some simple suggestions: a) I'm a fan of a patch-only evolution of code. All changes come from patches, and no changes come from anywhere else (eg. the committers IDE). Even if it is something as simple as cleaning up comments or changing a variable name. b) A patch gets applied if at least one reviewer +1s it, and no one -1s it. c) A patch should pass all unit tests. Any significant patch should come with additional unit tests. Some of this, of course, will mean more work for the committers. Sure, but such processes are essential if the project is to grow beyond a small group of core contributors. 3. Is trunk considered stable? Or aren't we missing a stable branch for the required stability? Once we have the separation between stable and trunk: Will patches really find it's way from trunk into stable? Is Facebook OK with that approach. Will everyone cope with the additional work of merging? Would it be useful ...or overkill to use merge tracking? I agree with Matt. Trunk should pass build + tests, but should not be trusted for production. I think 0.2 was supposed to be a stable branch. Avinash, Prashant -- what are your thoughts on this? Are you guys comfortable with this approach? Do you foresee any problems? Basically, use a release branch for production. The release branches only admit stability patches. New feature and cleanup patches go to trunk. Folks running Cassandra in production only need to be nervous when moving from one release to next, and not worry too much about every single patch breaking their running system. 4. Real world testing feedback is not publicly available. So the feedback on changes will only slowly reach the community. This is not easy for a project like this. But is there a faster way to provide testing feedback? (IIRC Yahoo was providing testing feedback for Hadoop. They even try to auto-apply patches from JIRA) With time, FB may be able to provide feedback from their divert some traffic to the new version system. Auto-applying patches from JIRA sounds a little ambitious right now :-) 5. Is there
Re: Development process (was: working together)
+1 On Thu, Apr 9, 2009 at 10:49 AM, Johan Oskarsson jo...@oskarsson.nu wrote: Thanks Sandeep. Would we all be comfortable adopting this process going forward, hopefully reducing friction, bugs and problems in general? I assume +1 from me and Sandeep so far. /Johan Sandeep Tata wrote: Johan, the wiki pages are great! I think they will help iron out our process for contributing and committing. (I added a pointer to the formatting conventions in HowToContribute , can't think of anything else to add) http://cwiki.apache.org/confluence/display/CSDR/HowToContribute http://cwiki.apache.org/confluence/display/CSDR/HowToCommit http://cwiki.apache.org/confluence/display/CSDR/HowToRelease A short summary and description of why these points make sense: * Patch-only evolution of code, attached to a jira issue * At least one +1 on each issue before it can be committed, -1 stops the patch. Those two points would make sure that if someone disagrees with a change, a refactoring etc, they have a chance to voice their opinion and steer it into the right direction. * Trunk is not considered stable, but must pass unit tests * Any non trivial change should include unit tests * When a branch is created to prepare for a release extra effort is put into QA to make sure the release is as stable as possible. Point releases would then go out to fix issues found after the release was done. * Once a release has been out for a while and people are using it in production without problems it is upgraded to stable status. The purpose of these points is to encourage a vibrant codebase, to not be afraid of for example refactoring if it improves the code readability or testability. I appreciate that Cassandra is a complex system and that changes might have unwanted side effects, but hopefully adding tests and code reviews will reduce those. As a final catch-all the release candidate and stable release process should help end users avoid bugs. Thoughts on the wiki pages? Do they help resolve some of the problems? /Johan Sandeep Tata wrote: Thoughts inline: So the problems I am seeing are: 1. We elected a committer without real community consensus. The barrier of entry was unnatural low on this one. On the other hand we need non-FB committers for the graduation. The more the better. (No reason for low entry barrier though!) I think everyone (including the FB guys) agree that Jonathan has been working hard to help move the codebase forward. He has been quick to revert changes that broke the code that the FB guys had in the pipeline and have committed since. I think much of the friction comes from not having a process, which takes us to Torsten's #2: 2. A missing definition of development process: - What is considered a valid code review? - How much are changes discussed up-front? - What is the roadmap? ...for whom? (weighted as a community) This is probably where we need most work. Here are some simple suggestions: a) I'm a fan of a patch-only evolution of code. All changes come from patches, and no changes come from anywhere else (eg. the committers IDE). Even if it is something as simple as cleaning up comments or changing a variable name. b) A patch gets applied if at least one reviewer +1s it, and no one -1s it. c) A patch should pass all unit tests. Any significant patch should come with additional unit tests. Some of this, of course, will mean more work for the committers. Sure, but such processes are essential if the project is to grow beyond a small group of core contributors. 3. Is trunk considered stable? Or aren't we missing a stable branch for the required stability? Once we have the separation between stable and trunk: Will patches really find it's way from trunk into stable? Is Facebook OK with that approach. Will everyone cope with the additional work of merging? Would it be useful ...or overkill to use merge tracking? I agree with Matt. Trunk should pass build + tests, but should not be trusted for production. I think 0.2 was supposed to be a stable branch. Avinash, Prashant -- what are your thoughts on this? Are you guys comfortable with this approach? Do you foresee any problems? Basically, use a release branch for production. The release branches only admit stability patches. New feature and cleanup patches go to trunk. Folks running Cassandra in production only need to be nervous when moving from one release to next, and not worry too much about every single patch breaking their running system. 4. Real world testing feedback is not publicly available. So the feedback on changes will only slowly reach the community. This is not easy for a project like this. But is there a faster way to provide testing feedback? (IIRC Yahoo was providing testing feedback for Hadoop. They even try to auto-apply patches from JIRA) With time, FB may be able to provide feedback from their divert some traffic to the new version system.
Re: Development process (was: working together)
On Thu, 2009-04-09 at 16:49 +0100, Johan Oskarsson wrote: Thanks Sandeep. Would we all be comfortable adopting this process going forward, hopefully reducing friction, bugs and problems in general? I assume +1 from me and Sandeep so far. +1 Sandeep Tata wrote: Johan, the wiki pages are great! I think they will help iron out our process for contributing and committing. (I added a pointer to the formatting conventions in HowToContribute , can't think of anything else to add) http://cwiki.apache.org/confluence/display/CSDR/HowToContribute http://cwiki.apache.org/confluence/display/CSDR/HowToCommit http://cwiki.apache.org/confluence/display/CSDR/HowToRelease -- Eric Evans eev...@rackspace.com
