Re: working together
On Tue, Apr 7, 2009 at 1:10 PM, Torsten Curdt tcu...@apache.org wrote: Hey folks, I thought back and forth about whether this email should go to the private list or not. But I have a Cocoon background and in Cocoon land we found it useful to only use the private list when there is really no other way. This is about the community so I think it deserves to be discussed in the open. I hope no one is offended by that approach. Thanks for doing it this way. Let me try to explain how I see the history of Cassandra. Please correct me where I am wrong: Cassandra was (mainly?) developed by Facebook and is being used in production over there. At some stage it got open sourced on Google Code. External contributors did not feel like there is a real community around it as patches did not get applied. Out of frustration people started (or at least proposed) to fork the code base. As a fork means community fragmentation the projects has turned to the Apache Incubator hoping to turn this fragmented contributors into a healthy community. In January the project got accepted. Papers were filed and in March we saw the code enter the Apache subversion repository. A few weeks later a first committer was proposed. Unfortunately no one noticed that the actual authors bringing the code were NOT on the private list where the vote was held. So we got a new committer without the consent and/or feedback of the original authors. A big surprise. I disagree here, I was fully aware that we hadn't formed the PPMC from the committership yet (read PPMC = mentors) and at least on my side it was by design. I should probably have pointed it out at the time just in case though, my bad. Avinash and Prashant are two very nice fellows but the Cassandra community has to be larger than FB employees and actually was already, given the different forks that were developed here and there. Waiting to form the PPMC until it's more balanced in this case seems like a reasonable thing to do. The people that brought over the code now feel a bit betrayed by the process. They have to deal with a committer that does changes all over the place on a code base they depend on for production. They have the feeling these changes are untested (at least not tested enough to get right into production at Facebook) and that this destabilize the code base. I wasn't aware of those feelings but I don't think they're justified, many different projects are in production in many different environments in the foundation and changes aren't a problem. That's what stable branches are for. Cassandra is now an Apache project, if it's successful it will be developed by a much larger community that it used to. That's the main goal, not a byproduct of incubation. What I would expect from committers more experienced in the codebase is guidance and transmission of expertise, not code freeze. On the other hand there is new blood, new drive to the project. While Facebook needs it's stability, other contributors needs the change to meet their goals and deadlines. So the problems I am seeing are: 1. We elected a committer without real community consensus. The barrier of entry was unnatural low on this one. On the other hand we need non-FB committers for the graduation. The more the better. (No reason for low entry barrier though!) If what you mean is that it was low because there was less direct peer review, I would agree (although I would consider Ian as a peer). If what you mean is that Jonathan hasn't enough merit to be a committer, I would disagree. Just have a look at his giithub branch :) 2. A missing definition of development process: - What is considered a valid code review? - How much are changes discussed up-front? - What is the roadmap? ...for whom? (weighted as a community) 3. Is trunk considered stable? Or aren't we missing a stable branch for the required stability? Once we have the separation between stable and trunk: Will patches really find it's way from trunk into stable? Is Facebook OK with that approach. Will everyone cope with the additional work of merging? Would it be useful ...or overkill to use merge tracking? A very useful tool here is what in Apache lingo is called lazy consensus. Anybody can build a proposition and submit it in the open. If nobody objects in a reasonable amount of time (usually 72 hours), then it's considered adopted. So I've read a couple of propositions from Jonathan on these points, why not formalizing that? 4. Real world testing feedback is not publicly available. So the feedback on changes will only slowly reach the community. This is not easy for a project like this. But is there a faster way to provide testing feedback? (IIRC Yahoo was providing testing feedback for Hadoop. They even try to auto-apply patches from JIRA) 5. Is there really no code ownership issue. Working on a code base for 1-2 years can get you attached to the code you have written.
Re: working together
just on a point here. They were invited from Day 1 (Actually 20-Jan-2009) to be on there. It wasn't done out of malice. On 08/04/2009, at 6:10 AM, Torsten Curdt wrote: Unfortunately no one noticed that the actual authors bringing the code were NOT on the private list where the vote was held. -- Ian Holsman i...@holsman.net
Re: working together
On Tue, Apr 7, 2009 at 5:39 PM, Ian Holsman i...@holsman.net wrote: just on a point here. They were invited from Day 1 (Actually 20-Jan-2009) to be on there. It wasn't done out of malice. So I'm slightly confused as to what the PPMC composition is now. On 08/04/2009, at 6:10 AM, Torsten Curdt wrote: Unfortunately no one noticed that the actual authors bringing the code were NOT on the private list where the vote was held. -- Ian Holsman i...@holsman.net
Re: working together
I don't think I ever spoke about anyone's skills ever over here. Avinash On Tue, Apr 7, 2009 at 9:12 PM, Ian Holsman i...@holsman.net wrote: Matt. please don't. your comments are just as valuable as anyone else's. just be aware that if you talk about someone's skills on this list, or their possible promotion to a committer it preempts other conversations happening, and can discourage that person from participating IF they are not made a committer then and there. many times (on other projects) people's names are put forth as possible committers (or members) and the feedback is 'not yet'. This can't be done on a public list. and why I am so discouraging of it on here. On 08/04/2009, at 2:06 PM, Matt Revelle wrote: On Apr 7, 2009, at 11:33 PM, Avinash Lakshman wrote: I think someone else opened that door here. Avinash I completely agree with Avinash that current committers need to be involved in the approval of new committers. There appears to have been a miscommunication that resulted in some committers not being subscribed to the private list. Since I'm not a contributing member of this project, I'll refrain from further comment. -- Ian Holsman i...@holsman.net
Re: working together
On Tue, Apr 7, 2009 at 8:11 PM, Avinash Lakshman avinash.laksh...@gmail.com wrote: snipped (2) This is something that I have said many times over. Certain things are the way they are for a reason. For example when I say ConcurrentHashMap is a memory hog I say it because we have seen this in practice. How does it manifest itself? I obviously do not recall since all this was over a year ago. No one can claim to have run tests the way we have in the last year and a half. One cannot just run some simple test and say well I do not see the problem. I am not dumb. Anyone having gone through the exercise of having built a system like this in an organization will realize that the tests are very intermingled with the organization's infrastructure. I have no time to rip that all apart and put together a test suite at this point. This is just an example. There are many such instances - after all - we are the ones who have the operational experience with this and I do not think anyone can claim to understand the behavior this system in production workloads better than we do. Look, what you're saying here is basically we know better and you're stupid, so don't touch our code and don't ask questions, we can't provide answers anyway. I'm hoping that's not the way you meant it (emails do that) but that's the essence of what came across. You just can't run an open source project by saying this on its development list. Matthieu My understanding was that new committers come in and start with some feature implement that and then slowly start looking into what more they could do going forward. It is NOT come in and refactor the hell out of the system because you like something to be in a specific way. I do not beleive this will fly in any community. It is something like we now going through the entire code base and changing all the stuff just because I like it in a specific way. This seems ludicrous. We may have no experience in open source but we understand etiquette very well. This just doesn't seem the way things work in other Apache projects which are successful. We work very closely with two committers from the Hadoop project who were flabbergasted with the refactor changes that were going in. That is my gripe with the whole thing. Cheers Avinash On Tue, Apr 7, 2009 at 7:30 PM, Jonathan Ellis jbel...@gmail.com wrote: On Tue, Apr 7, 2009 at 3:10 PM, Torsten Curdt tcu...@apache.org wrote: So the problems I am seeing are: 1. We elected a committer without real community consensus. The barrier of entry was unnatural low on this one. On the other hand we need non-FB committers for the graduation. The more the better. (No reason for low entry barrier though!) It's unfortunate that Avinash and Prashant weren't part of the process. Still, when I talked to Avinash on March 1, he told me [and this is a direct quote] If I had known you earlier I would have added you as a committer. So when I asked one of the mentors how to become a committer and it worked out from there it did not occur to me that anything was wrong. 2. A missing definition of development process: - What is considered a valid code review? - How much are changes discussed up-front? I think we have a handle on this now. All changes are put on Jira for review and are not committed until there is at least one +1 from a reviewer. (I personally prefer post-commit review because manually attaching and applying patches is tedious but we don't have enough people following the commit log for that to work right now.) - What is the roadmap? ...for whom? (weighted as a community) That's worth a separate thread. Such as this one. :) http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg00160.html 3. Is trunk considered stable? Or aren't we missing a stable branch for the required stability? Once we have the separation between stable and trunk: Will patches really find it's way from trunk into stable? Is Facebook OK with that approach. Will everyone cope with the additional work of merging? Would it be useful ...or overkill to use merge tracking? I'm happy to assist with merging code to or from stable branches in this scenario. This is a tough situation but I hope everyone sees this as an opportunity. Please let's discuss this openly in civilize manner. Focusing on how to solve these points rather than looking at the past. Please talk to each other. Can you/we work this out together? This can still be a win/win for everyone. I think that historically facebook has felt like the community hasn't contributed much of value, but we're starting to change that. The build and test process is dramatically better than it was before thanks to community contributions. We have a real daemon mode. (Well, not in the purest sense, but it runs in the background nicely w/o nohup or screen. :) We've
Re: working together
(2) This is something that I have said many times over. Certain things are the way they are for a reason. For example when I say ConcurrentHashMap is a memory hog I say it because we have seen this in practice. How does it manifest itself? I obviously do not recall since all this was over a year ago. Not to sidetrack the conversation too much -- but this is a perfect example of why I suggested more informative changelog entries would be valuable to the project going forward. If you record this sort of information along with changes to a file, then someone wondering why code is written in a certain way at least has a hope of using svn blame or the like and actually finding an answer. - R.
