Re: Flume Graduation (was Re: June reports in two weeks)
On Tue, Jun 5, 2012 at 2:22 PM, Franklin, Matthew B. mfrank...@mitre.org wrote: As not everyone has read or is privy to the private list discussions, what they see is that you have a standing -1 vote from a mentor who has expressed concerns on this list and the dev list as to whether or not the diversity requirement has been met. Thanks, Matt, that is exactly what I perceived. I had gone back and scanned through the Flume dev list for the previous month before posting, too. :( It looks (and looked to me initially) like an incubator project that is dominated by a single company was trying to push itself through the incubation process at whatever cost and before it was ready; something that most IPMC members would likely resist. Everything I posted was written under the assumption that all contributors were participating as individuals in good faith and without any undue company influence. I'm only human, though. Without the context of that private thread, Patrick Hunt's assurances that Extensive discussion ensued and afaict Ralph's concerns were addressed made no sense -- Ralph had stated mere moments before on this list that if the basis we are to use is whether a single company or entity is vital to a project then I don't believe Flume is quite there. And then there were the IMO unconvincing lawyerly statistical arguments which had been advanced that the [now vacated] diversity requirement had been met. I had to keep struggling to understand what on earth could be the explanation. If this type of misperception happened at the incubator, what does the foundation and flume community see? With the additional background I have read, it seems that the community is much healthier than it appears and is acting on a lot of communication and consensus that was driven on the private list. IMO, it would have been better to ask Ralph if he was comfortable retracting his -1 prior to pushing the general@ VOTE and if he was not, then participate in the diversity discussion on general@ until some consensus has been reached. +1 What a frustrating waste of time this has been. Marvin Humphrey - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
On Sat, May 26, 2012 at 11:44 PM, Ralph Goers ralph.go...@dslextreme.com wrote: Another way of looking at these same statistics: Cloudera - 217 Other - 16 That means Cloudera is responsible for over 93% of the Jira issues. It is great that Cloudera is doing so much work but those stats hardly prove diversity. I was surprised to see the IPMC Flume graduation VOTE today -- I don't recall seeing another situation like it in the last couple years, where the community graduation VOTE was contended. I checked the Flume dev list archives and I don't see a message from Ralph indicating that he thinks the latest measures address the concerns that have been raised. Last September, Jukka wrote this on the ManifoldCF dev list: http://s.apache.org/community-lessons-from-tika To put things in perspective, since the beginning of this year Karl has made over 96% of all ManifoldCF commits. This makes the bus factor [1] of the project pretty high, and suggests that a more diverse development community is needed. The solution is not to have Karl commit less, but to get other people to more actively join the fun. The situation here is roughly similar to what we experienced during the incubation of Apache Tika. In the last year before graduation (2008) I was responsible for about 87% of all commits, which raised similar concerns about diversity [2]. The solution then was to graduate into a Lucene subproject instead of a full TLP, so that the larger project could still provide oversight and continuity in case things went wrong. Since then Lucene has shed out most subprojects to avoid being too large to manage, and by the time Tika in 2010 became a TLP by itself my share of all commits had shrunk to a still high but much more reasonable 62%. Today I'm still the most active committer, but my share of all the activity is down to 44%. What are the percentages we'd like to see with Flume? Marvin Humphrey - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
On Jun 5, 2012, at 2:19 PM, Marvin Humphrey wrote: On Sat, May 26, 2012 at 11:44 PM, Ralph Goers ralph.go...@dslextreme.com wrote: Another way of looking at these same statistics: Cloudera - 217 Other - 16 That means Cloudera is responsible for over 93% of the Jira issues. It is great that Cloudera is doing so much work but those stats hardly prove diversity. I was surprised to see the IPMC Flume graduation VOTE today -- I don't recall seeing another situation like it in the last couple years, where the community graduation VOTE was contended. I checked the Flume dev list archives and I don't see a message from Ralph indicating that he thinks the latest measures address the concerns that have been raised. Agreed. It's hard to vote for graduation for a podling when one of the mentors feels strongly that the podling is not ready. Alan. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
The graduation requirements say The project is considered to have a diverse community when it is not highly dependent on any single contributor (there are at least 3 legally independent committers and there is no single company or entity that is vital to the success of the project). Basically this means that when a project mostly consists of contributors from one company, this is a sign of not being diverse enough.. This doesn't specify a hard number. In fact, Roy responded to this thread saying he doesn't believe there even is a diversity requirement - There is no diversity requirement for graduating from the incubator. In many ways, incubation hinders community growth. The requirement is that the project makes decisions as an Apache project, not in private, which is harder to get used to doing if a lot of people share the same office. So I am left a bit confused. If I go by the what the graduation page says literally, then all the statistics that have been generated would seem to show that Cloudera is vital to the success of the project. Although Arvind is a bit of the driving force, I'm sure if something terrible were to happen to him Cloudera would insure his energy was replaced. However, if something terrible happened to Cloudera I suspect we would have several Apache projects in trouble, not just Flume. While I clearly don't like some of the ways the project has chosen to organize itself, all those decisions were done properly and in public. Again, while I don't like that little discussion happens on the dev list, it does happen in Jira issues and in the review board, all of which is routed to the dev list, so again, most, if not all, of the development is done in public. So my answer to the question is really that I am finding it hard to reconcile whether we actually have or should have a diversity requirement. From what I've been told privately Flume would certainly not be the first project to graduate from the incubator in a similar situation. The other thing I find interesting is that I am also the only non-Cloudera mentor on the project. I find it a bit odd that while the incubator has the requirement for graduation it doesn't have any such requirement for a codling's mentors. That said, IMO every one of the mentors on the project has been doing a good job. One other disclaimer. My employer is a customer of Cloudera specifically for paid support for Flume, so I also have a vested interest in seeing both the project and Cloudera succeed. However, with regards to Flume's graduation, I haven't even discussed this issue with anyone in by $dayjob. So again - if the basis we are to use is whether a single company or entity is vital to a project then I don't believe Flume is quite there. OTOH I am not completely necessary that that is vital for graduation, in which case the section in the graduation requirements needs to be changed. So at this point the best I can do is say I'm not really sure how to vote. Ralph On Jun 5, 2012, at 6:49 AM, Alan Gates wrote: On Jun 5, 2012, at 2:19 PM, Marvin Humphrey wrote: On Sat, May 26, 2012 at 11:44 PM, Ralph Goers ralph.go...@dslextreme.com wrote: Another way of looking at these same statistics: Cloudera - 217 Other - 16 That means Cloudera is responsible for over 93% of the Jira issues. It is great that Cloudera is doing so much work but those stats hardly prove diversity. I was surprised to see the IPMC Flume graduation VOTE today -- I don't recall seeing another situation like it in the last couple years, where the community graduation VOTE was contended. I checked the Flume dev list archives and I don't see a message from Ralph indicating that he thinks the latest measures address the concerns that have been raised. Agreed. It's hard to vote for graduation for a podling when one of the mentors feels strongly that the podling is not ready. Alan. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
Isn't this why we vote. To come to a decision when consensus can't be reached and allow people to move on. Patrick On Tue, Jun 5, 2012 at 8:22 AM, Ralph Goers ralph.go...@dslextreme.com wrote: The graduation requirements say The project is considered to have a diverse community when it is not highly dependent on any single contributor (there are at least 3 legally independent committers and there is no single company or entity that is vital to the success of the project). Basically this means that when a project mostly consists of contributors from one company, this is a sign of not being diverse enough.. This doesn't specify a hard number. In fact, Roy responded to this thread saying he doesn't believe there even is a diversity requirement - There is no diversity requirement for graduating from the incubator. In many ways, incubation hinders community growth. The requirement is that the project makes decisions as an Apache project, not in private, which is harder to get used to doing if a lot of people share the same office. So I am left a bit confused. If I go by the what the graduation page says literally, then all the statistics that have been generated would seem to show that Cloudera is vital to the success of the project. Although Arvind is a bit of the driving force, I'm sure if something terrible were to happen to him Cloudera would insure his energy was replaced. However, if something terrible happened to Cloudera I suspect we would have several Apache projects in trouble, not just Flume. While I clearly don't like some of the ways the project has chosen to organize itself, all those decisions were done properly and in public. Again, while I don't like that little discussion happens on the dev list, it does happen in Jira issues and in the review board, all of which is routed to the dev list, so again, most, if not all, of the development is done in public. So my answer to the question is really that I am finding it hard to reconcile whether we actually have or should have a diversity requirement. From what I've been told privately Flume would certainly not be the first project to graduate from the incubator in a similar situation. The other thing I find interesting is that I am also the only non-Cloudera mentor on the project. I find it a bit odd that while the incubator has the requirement for graduation it doesn't have any such requirement for a codling's mentors. That said, IMO every one of the mentors on the project has been doing a good job. One other disclaimer. My employer is a customer of Cloudera specifically for paid support for Flume, so I also have a vested interest in seeing both the project and Cloudera succeed. However, with regards to Flume's graduation, I haven't even discussed this issue with anyone in by $dayjob. So again - if the basis we are to use is whether a single company or entity is vital to a project then I don't believe Flume is quite there. OTOH I am not completely necessary that that is vital for graduation, in which case the section in the graduation requirements needs to be changed. So at this point the best I can do is say I'm not really sure how to vote. Ralph On Jun 5, 2012, at 6:49 AM, Alan Gates wrote: On Jun 5, 2012, at 2:19 PM, Marvin Humphrey wrote: On Sat, May 26, 2012 at 11:44 PM, Ralph Goers ralph.go...@dslextreme.com wrote: Another way of looking at these same statistics: Cloudera - 217 Other - 16 That means Cloudera is responsible for over 93% of the Jira issues. It is great that Cloudera is doing so much work but those stats hardly prove diversity. I was surprised to see the IPMC Flume graduation VOTE today -- I don't recall seeing another situation like it in the last couple years, where the community graduation VOTE was contended. I checked the Flume dev list archives and I don't see a message from Ralph indicating that he thinks the latest measures address the concerns that have been raised. Agreed. It's hard to vote for graduation for a podling when one of the mentors feels strongly that the podling is not ready. Alan. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
Not really. Generally it is best for a vote to be a proof of consensus, especially for bigger topics (like graduation). And I certainly would add a -1 vote were a project attempting to override a mentor. Upayavira On Tue, Jun 5, 2012, at 09:53 AM, Patrick Hunt wrote: Isn't this why we vote. To come to a decision when consensus can't be reached and allow people to move on. Patrick On Tue, Jun 5, 2012 at 8:22 AM, Ralph Goers ralph.go...@dslextreme.com wrote: The graduation requirements say The project is considered to have a diverse community when it is not highly dependent on any single contributor (there are at least 3 legally independent committers and there is no single company or entity that is vital to the success of the project). Basically this means that when a project mostly consists of contributors from one company, this is a sign of not being diverse enough.. This doesn't specify a hard number. In fact, Roy responded to this thread saying he doesn't believe there even is a diversity requirement - There is no diversity requirement for graduating from the incubator. In many ways, incubation hinders community growth. The requirement is that the project makes decisions as an Apache project, not in private, which is harder to get used to doing if a lot of people share the same office. So I am left a bit confused. If I go by the what the graduation page says literally, then all the statistics that have been generated would seem to show that Cloudera is vital to the success of the project. Although Arvind is a bit of the driving force, I'm sure if something terrible were to happen to him Cloudera would insure his energy was replaced. However, if something terrible happened to Cloudera I suspect we would have several Apache projects in trouble, not just Flume. While I clearly don't like some of the ways the project has chosen to organize itself, all those decisions were done properly and in public. Again, while I don't like that little discussion happens on the dev list, it does happen in Jira issues and in the review board, all of which is routed to the dev list, so again, most, if not all, of the development is done in public. So my answer to the question is really that I am finding it hard to reconcile whether we actually have or should have a diversity requirement. From what I've been told privately Flume would certainly not be the first project to graduate from the incubator in a similar situation. The other thing I find interesting is that I am also the only non-Cloudera mentor on the project. I find it a bit odd that while the incubator has the requirement for graduation it doesn't have any such requirement for a codling's mentors. That said, IMO every one of the mentors on the project has been doing a good job. One other disclaimer. My employer is a customer of Cloudera specifically for paid support for Flume, so I also have a vested interest in seeing both the project and Cloudera succeed. However, with regards to Flume's graduation, I haven't even discussed this issue with anyone in by $dayjob. So again - if the basis we are to use is whether a single company or entity is vital to a project then I don't believe Flume is quite there. OTOH I am not completely necessary that that is vital for graduation, in which case the section in the graduation requirements needs to be changed. So at this point the best I can do is say I'm not really sure how to vote. Ralph On Jun 5, 2012, at 6:49 AM, Alan Gates wrote: On Jun 5, 2012, at 2:19 PM, Marvin Humphrey wrote: On Sat, May 26, 2012 at 11:44 PM, Ralph Goers ralph.go...@dslextreme.com wrote: Another way of looking at these same statistics: Cloudera - 217 Other - 16 That means Cloudera is responsible for over 93% of the Jira issues. It is great that Cloudera is doing so much work but those stats hardly prove diversity. I was surprised to see the IPMC Flume graduation VOTE today -- I don't recall seeing another situation like it in the last couple years, where the community graduation VOTE was contended. I checked the Flume dev list archives and I don't see a message from Ralph indicating that he thinks the latest measures address the concerns that have been raised. Agreed. It's hard to vote for graduation for a podling when one of the mentors feels strongly that the podling is not ready. Alan. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
I suppose. But i guess I'd rather vote on whether diversity is a requirement for graduation before I voted on the graduation itself. Ralph On Jun 5, 2012, at 9:53 AM, Patrick Hunt wrote: Isn't this why we vote. To come to a decision when consensus can't be reached and allow people to move on. Patrick On Tue, Jun 5, 2012 at 8:22 AM, Ralph Goers ralph.go...@dslextreme.com wrote: The graduation requirements say The project is considered to have a diverse community when it is not highly dependent on any single contributor (there are at least 3 legally independent committers and there is no single company or entity that is vital to the success of the project). Basically this means that when a project mostly consists of contributors from one company, this is a sign of not being diverse enough.. This doesn't specify a hard number. In fact, Roy responded to this thread saying he doesn't believe there even is a diversity requirement - There is no diversity requirement for graduating from the incubator. In many ways, incubation hinders community growth. The requirement is that the project makes decisions as an Apache project, not in private, which is harder to get used to doing if a lot of people share the same office. So I am left a bit confused. If I go by the what the graduation page says literally, then all the statistics that have been generated would seem to show that Cloudera is vital to the success of the project. Although Arvind is a bit of the driving force, I'm sure if something terrible were to happen to him Cloudera would insure his energy was replaced. However, if something terrible happened to Cloudera I suspect we would have several Apache projects in trouble, not just Flume. While I clearly don't like some of the ways the project has chosen to organize itself, all those decisions were done properly and in public. Again, while I don't like that little discussion happens on the dev list, it does happen in Jira issues and in the review board, all of which is routed to the dev list, so again, most, if not all, of the development is done in public. So my answer to the question is really that I am finding it hard to reconcile whether we actually have or should have a diversity requirement. From what I've been told privately Flume would certainly not be the first project to graduate from the incubator in a similar situation. The other thing I find interesting is that I am also the only non-Cloudera mentor on the project. I find it a bit odd that while the incubator has the requirement for graduation it doesn't have any such requirement for a codling's mentors. That said, IMO every one of the mentors on the project has been doing a good job. One other disclaimer. My employer is a customer of Cloudera specifically for paid support for Flume, so I also have a vested interest in seeing both the project and Cloudera succeed. However, with regards to Flume's graduation, I haven't even discussed this issue with anyone in by $dayjob. So again - if the basis we are to use is whether a single company or entity is vital to a project then I don't believe Flume is quite there. OTOH I am not completely necessary that that is vital for graduation, in which case the section in the graduation requirements needs to be changed. So at this point the best I can do is say I'm not really sure how to vote. Ralph On Jun 5, 2012, at 6:49 AM, Alan Gates wrote: On Jun 5, 2012, at 2:19 PM, Marvin Humphrey wrote: On Sat, May 26, 2012 at 11:44 PM, Ralph Goers ralph.go...@dslextreme.com wrote: Another way of looking at these same statistics: Cloudera - 217 Other - 16 That means Cloudera is responsible for over 93% of the Jira issues. It is great that Cloudera is doing so much work but those stats hardly prove diversity. I was surprised to see the IPMC Flume graduation VOTE today -- I don't recall seeing another situation like it in the last couple years, where the community graduation VOTE was contended. I checked the Flume dev list archives and I don't see a message from Ralph indicating that he thinks the latest measures address the concerns that have been raised. Agreed. It's hard to vote for graduation for a podling when one of the mentors feels strongly that the podling is not ready. Alan. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional
Re: Flume Graduation (was Re: June reports in two weeks)
On Tue, Jun 5, 2012 at 9:53 AM, Patrick Hunt ph...@apache.org wrote: Isn't this why we vote. To come to a decision when consensus can't be reached and allow people to move on. When diversity concerns were raised in the ManifoldCF podling by Jukka, graduation was delayed to address them. When diversity concerns were raised in the Lucy podling by Torsten Curdt, graduation was delayed to address them. When diversity concerns were raised with regards to Flume, a VOTE was called. Contentious VOTEs force people to take sides and create winners and losers where none existed before. It is often worth going the extra mile to avoid imposing a tyranny of the majority and fostering alienation. In my opinion, there would be a benefit to the Flume community for staying in the Incubator a while longer and wrestling with how to increase diversity. Have you folks ever had a non-Cloudera Release Manager? Was there ever a possibility of a non-Cloudera Flume VP? How about waiting until someone other than a Cloudera person emerges who is willing to lead a graduation push? I think the community would develop healthy habits by prioritizing such concerns for a little while. On the other hand, there are benefits to graduation in terms of enlarging the pool of potential contributors by those who are willing to get involved with graduated projects but not podlings. Ralph has shown an open mind and has been weighing the plusses and minuses. We all want what is best for Flume. It wasn't necessary to call a VOTE and override a Mentor. There were other ways to resolve the issue. Flume chose the contentious route, and that bothers me. I don't think it bodes well. Marvin Humphrey - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
On Tue, Jun 5, 2012 at 11:01 AM, Marvin Humphrey mar...@rectangular.com wrote: On Tue, Jun 5, 2012 at 9:53 AM, Patrick Hunt ph...@apache.org wrote: Isn't this why we vote. To come to a decision when consensus can't be reached and allow people to move on. When diversity concerns were raised in the ManifoldCF podling by Jukka, graduation was delayed to address them. When diversity concerns were raised in the Lucy podling by Torsten Curdt, graduation was delayed to address them. When diversity concerns were raised with regards to Flume, a VOTE was called. I think that's an oversimplification that ignores the work and good faith the Flume community has done to address the concerns raised. Extensive discussion ensued and afaict Ralph's concerns were addressed. He raised issues and the community worked to resolve them. (ie promoting committers to pmc as part of graduation). Contentious VOTEs force people to take sides and create winners and losers where none existed before. It is often worth going the extra mile to avoid imposing a tyranny of the majority and fostering alienation. In my opinion, there would be a benefit to the Flume community for staying in the Incubator a while longer and wrestling with how to increase diversity. Have you folks ever had a non-Cloudera Release Manager? Was there ever a possibility of a non-Cloudera Flume VP? How about waiting until someone other than a Cloudera person emerges who is willing to lead a graduation push? I think the community would develop healthy habits by prioritizing such concerns for a little while. On the other hand, there are benefits to graduation in terms of enlarging the pool of potential contributors by those who are willing to get involved with graduated projects but not podlings. Ralph has shown an open mind and has been weighing the plusses and minuses. We all want what is best for Flume. All great ideas. But this highlights my concern and the problem I have with many of these discussions. I've been on the other side of the fence on this stuff. You're trying to make forward progress. How do you separate the must do's from the good ideas. From what I could see the Flume community worked with the IPMC to review the status and address any concerns. At which point the vote was moved forward. If you don't agree then -1. That's why we have voting. That's why it's majority vs veto'able. If the vote doesn't pass is will list explicit grounds for a -1, the community can go back, address those, and start a new vote when the issues are addressed. There should be no shame in getting a -1, voting (after healthy discussion) allows the community to speak clearly and definitively and it also respects the community requesting the vote. Patrick - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
On Jun 5, 2012, at 2:22 PM, Franklin, Matthew B. wrote: Based on some digging, I think what you are mostly facing is a battle of perception. As not everyone has read or is privy to the private list discussions, what they see is that you have a standing -1 vote from a mentor who has expressed concerns on this list and the dev list as to whether or not the diversity requirement has been met. It looks (and looked to me initially) like an incubator project that is dominated by a single company was trying to push itself through the incubation process at whatever cost and before it was ready; something that most IPMC members would likely resist. If this type of misperception happened at the incubator, what does the foundation and flume community see? With the additional background I have read, it seems that the community is much healthier than it appears and is acting on a lot of communication and consensus that was driven on the private list. IMO, it would have been better to ask Ralph if he was comfortable retracting his -1 prior to pushing the general@ VOTE and if he was not, then participate in the diversity discussion on general@ until some consensus has been reached. I posted an email earlier today where I discussed my confusion over the diversity requirement. I'm not comfortable doing anything without getting some feedback on whether the diversity requirement, as currently stated on the wiki, is correct or whether diversity should simply be measured by how a project makes its decisions as Roy suggests. Ralph - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
Hi, On Tue, Jun 5, 2012 at 11:45 PM, Ralph Goers ralph.go...@dslextreme.com wrote: I posted an email earlier today where I discussed my confusion over the diversity requirement. I'm not comfortable doing anything without getting some feedback on whether the diversity requirement, as currently stated on the wiki, is correct or whether diversity should simply be measured by how a project makes its decisions as Roy suggests. All the graduation criteria are things that the IPMC has previously considered important things to consider when deciding if a community has reached maturity as defined in the original Incubator resolution [1]. The criteria have evolved over time and will continue to do so. That said, I think diversity is an important factor of community maturity and should definitely be considered when casting a vote on the graduation of a project. The traditional metric of at least three independent (active) committers is a good guideline, though it has often especially with smaller projects been judged subjectively and weighed in relation to other aspects of community health. As for Flume, you and the other mentors are in the best position to consider the overall maturity of the project. Is the project ready to function as a standalone TLP on it's own according to the Apache Way and the ASF policies, or is there still something that the Incubator can or should teach the community? Personally, based on a few closer looks at Flume, it seems to me that the community passes that barrier and I'm satisfied with the way they've decided to addressed the concerns that were raised. But before casting my vote I'd love to hear your opinion on this since you raised the issue originally and as a mentor you have much better insight on the actual community dynamics within Flume. [1] http://apache.org/foundation/records/minutes/2002/board_minutes_2002_10_16.txt BR, Jukka Zitting - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
On Jun 5, 2012, at 4:10 PM, Jukka Zitting wrote: Hi, On Tue, Jun 5, 2012 at 11:45 PM, Ralph Goers ralph.go...@dslextreme.com wrote: I posted an email earlier today where I discussed my confusion over the diversity requirement. I'm not comfortable doing anything without getting some feedback on whether the diversity requirement, as currently stated on the wiki, is correct or whether diversity should simply be measured by how a project makes its decisions as Roy suggests. All the graduation criteria are things that the IPMC has previously considered important things to consider when deciding if a community has reached maturity as defined in the original Incubator resolution [1]. The criteria have evolved over time and will continue to do so. That said, I think diversity is an important factor of community maturity and should definitely be considered when casting a vote on the graduation of a project. The traditional metric of at least three independent (active) committers is a good guideline, though it has often especially with smaller projects been judged subjectively and weighed in relation to other aspects of community health. As for Flume, you and the other mentors are in the best position to consider the overall maturity of the project. Is the project ready to function as a standalone TLP on it's own according to the Apache Way and the ASF policies, or is there still something that the Incubator can or should teach the community? Personally, based on a few closer looks at Flume, it seems to me that the community passes that barrier and I'm satisfied with the way they've decided to addressed the concerns that were raised. But before casting my vote I'd love to hear your opinion on this since you raised the issue originally and as a mentor you have much better insight on the actual community dynamics within Flume. [1] http://apache.org/foundation/records/minutes/2002/board_minutes_2002_10_16.txt Thanks. I think your view here aligns with the way I would prefer to evaluate projects. As to whether I believe Flume is ready to function as a standalone TLP - absolutely yes. As I have said, the only criteria it doesn't meet is the statement that no single company or entity is vital to the success of the project. I believe the project is still highly dependent on Cloudera. But I'm ready to vote +1 if that is not considered to be an absolute requirement for graduation. Ralph - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
Closing the loop on this thread: The Flume PPMC has discussed the diversity issue and have concluded to put all current committers as PMC when drafting the resolution. Also, the wiki has been updated with details on how to contribute and commit. https://cwiki.apache.org/confluence/display/FLUME/How+to+Contribute https://cwiki.apache.org/confluence/display/FLUME/How+to+Commit Thanks, Arvind Prabhakar On Sun, May 27, 2012 at 3:21 AM, Arvind Prabhakar arv...@apache.org wrote: Thanks Jukka for your suggestions. 1. Regarding wiki - I have taken a note of that and will update it soon. 2. Regarding doing away with the difference between PPMC and committers, I am told that other projects do this during graduation. I.e., they promote all existing committers to PMC status during graduation. If we do that, the diversity concern will be addressed and we can then debate the bylaws once the PMC is formed. Thanks, Arvind Prabhakar On Sun, May 27, 2012 at 2:40 AM, Jukka Zitting jukka.zitt...@gmail.comwrote: Hi, On Sun, May 27, 2012 at 6:29 AM, Arvind Prabhakar arv...@apache.org wrote: As you noted in your comments above - the Flume project tends to follow RTC with the reviewer committing the code. I happen to have taken up the role of the reviewer for the most part and hence you see the skewed commit counts. It might be useful if you for example expanded the How to Commit wiki page [1] with a better description of what a reviewer should take in to account when committing reviewed changes. Things that might be obvious to you and others who've been around for longer, but not necessarily for newcomers. The more people you have committing code, the less the project is dependent on the silent knowledge of any single contributor. On Sat, May 26, 2012 at 4:43 PM, Jukka Zitting jukka.zitt...@gmail.com wrote: Do you think this is a problem for the community? If yes, how do you plan to fix it? If not, why? I don't think this is a problem because while most Cloudera committers have the luxury of working on the project during regular working hours, others do that during their off hours. Hence the majority of contributions coming from Cloudera. OK, fair enough. Such a scenario exists or has existed also in other Apache projects like Jackrabbit that I'm most familiar with. It can be a tricky balance to maintain a level playing field in such cases, for example by making sure that all relevant project discussions happen out in the open and that some committers don't feel like junior partners with less ability to influence the project. It sounds like you have a reasonably good handle on that, so I'm not too worried, but my instinct suggests that the strict RTC model and distinction between committers and (P)PMC members may be structural factors that could easily end up tripping that balance. Are these really essential tools for the project or could you live without them? Other solutions to the RTC model include separate maintenance branches with stricter review and testing requirements, and the only cases where I really see a need for the committer/(P)PMC separation is with umbrella projects or special cases like GSoC students or co-operation across project boundaries. I know you have good arguments for the way things work in Flume now, and ultimately you're the ones to decide how you want to work. So take the above just as friendly suggestions for things you may want to consider. Whatever the outcome, it would be good for Flume to document such decisions and the rationale behind them as a set of project bylaws. This is what the bylaws section of the graduation resolution is about: RESOLVED, that the initial Apache $project PMC be and hereby is tasked with the creation of a set of bylaws intended to encourage open development and increased participation in the Apache $foo Project; and be it further [1] https://cwiki.apache.org/confluence/display/FLUME/How+to+Commit BR, Jukka Zitting - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
Doesn't the IPMC (and not any individuals, even Roy) decide what the official Incubation policies should be? Ralph's reply, and my general reading of the feeling in the IPMC is that graduation votes *should* take affiliation diversity into account. Note that some could argue Flume technically meets the diversity criteria, however I really like Jukka's analysis of who's actually making the commits - which means they may not necessarily meet the diversity criteria in spirit yet. - Shane P.S. For our public readers, note that Roy's emails are always worth reading. On 2012-05-27 5:17 AM, Ralph Goers wrote: Roy, What you are saying directly contradicts http://incubator.apache.org/guides/graduation.html#community, especially the statement Basically this means that when a project mostly consists of contributors from one company, this is a sign of not being diverse enough. Now, if that page is wrong and diversity is not a requirement for graduation then, of course, that would certainly remove any objections I have for Flume's graduation. However, I've heard it said that the ASF does not want to simply provide the infrastructure for companies to host their products on. Ralph On May 27, 2012, at 1:35 AM, Roy T. Fielding wrote: There is no diversity requirement for graduating from the incubator. In many ways, incubation hinders community growth. The requirement is that the project makes decisions as an Apache project, not in private, which is harder to get used to doing if a lot of people share the same office. Diversity is only a warning sign that means we need to check for decisions made in our forums and advise accordingly. It is not an end in itself, nor has lack of diversity hindered other projects from continuing on to build a larger community as a TLP. Roy On May 26, 2012, at 11:44 PM, Ralph Goersralph.go...@dslextreme.com wrote: On May 26, 2012, at 9:29 PM, Arvind Prabhakar wrote: Hi Jukka, On Sat, May 26, 2012 at 4:43 PM, Jukka Zittingjukka.zitt...@gmail.comwrote: IIUC Flume operates under an RTC model where people are not supposed to commit their own changes, which obviously makes the above data less relevant for evaluating the true diversity of the community. However, seeing only a single trivial commit by both jarcec and juhanic even though they became committers already over three months ago does seem to suggest that they may not be as comfortable in their committer role as people from Cloudera are. As you noted in your comments above - the Flume project tends to follow RTC with the reviewer committing the code. I happen to have taken up the role of the reviewer for the most part and hence you see the skewed commit counts. If you want to see the actual contribution, I would suggest looking at fixed JIRA issues by assignees. A quick report yields the following: aprabhkar - 26 - Cloudera [6] brocknoland - 19 - Cloudera [7] esammer - 56 - Cloudera [8] hshreedharan - 34 - Cloudera [9] jarcec - 6 - AVG Technologies [10] jmhsieh - 8 - Cloudera [11] juhanic - 9 - CyberAgent [12] mpercy - 34 - Cloudera [13] m...@apache.org - 1 - Trend Micro [14] prasadm - 34 - Cloudera [15] t...@cloudera.com - 3 - Cloudera [16] w...@cloudera.com - 3 - Cloudera [17] Looking at this, the average number of issues resolved by Cloudera committers (not counting Tom who is a mentor) is 26, and that for non-Cloudera committers is 5. Note that this number does not include other committer work such as the number of code reviews they have done, the number of design discussions they have participated in, something that is very valuable to the project. Another way of looking at these same statistics: Cloudera - 217 Other - 16 That means Cloudera is responsible for over 93% of the Jira issues. It is great that Cloudera is doing so much work but those stats hardly prove diversity. Ralph - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
Shane Curcuru wrote on Tue, May 29, 2012 at 07:13:29 -0400: Doesn't the IPMC (and not any individuals, even Roy) decide what the official Incubation policies should be? Ralph's reply, and my Well, yes, but the IPMC can only decide whether or not to recommend graduation. The Board can graduate a project despite a recommendation to the contrary from the IPMC. Daniel general reading of the feeling in the IPMC is that graduation votes *should* take affiliation diversity into account. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
On Tue, May 29, 2012 at 1:20 PM, Daniel Shahaf d...@daniel.shahaf.name wrote: Shane Curcuru wrote on Tue, May 29, 2012 at 07:13:29 -0400: Doesn't the IPMC (and not any individuals, even Roy) decide what the official Incubation policies should be?... Well, yes, but the IPMC can only decide whether or not to recommend graduation. The Board can graduate a project despite a recommendation to the contrary from the IPMC In theory yes, but I don't think that has ever happened. AFAIK the current board wants projects to take care of their own destiny as much as possible, and that includes the Incubator. -Bertrand - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
From a mobile device - forgive errors and terseness On May 29, 2012 12:14 PM, Shane Curcuru a...@shanecurcuru.org wrote: Doesn't the IPMC (and not any individuals, even Roy) decide what the official Incubation policies should be? Ralph's reply, and my general reading of the feeling in the IPMC is that graduation votes *should* take affiliation diversity into account. +1 on both counts. For me what is important is not so much diversity in numbers but diversity in engagement, particularly in decision making. I've not reviewed Flume myself so I have to listen to mentors. What I hear is a concern about numbers, but no explicit concern about decision making. Note that some could argue Flume technically meets the diversity criteria, however I really like Jukka's analysis of who's actually making the commits - which means they may not necessarily meet the diversity criteria in spirit yet. Although when the RTC process is considered this point becomes less valid. Ross - Shane P.S. For our public readers, note that Roy's emails are always worth reading. On 2012-05-27 5:17 AM, Ralph Goers wrote: Roy, What you are saying directly contradicts http://incubator.apache.org/guides/graduation.html#community, especially the statement Basically this means that when a project mostly consists of contributors from one company, this is a sign of not being diverse enough. Now, if that page is wrong and diversity is not a requirement for graduation then, of course, that would certainly remove any objections I have for Flume's graduation. However, I've heard it said that the ASF does not want to simply provide the infrastructure for companies to host their products on. Ralph On May 27, 2012, at 1:35 AM, Roy T. Fielding wrote: There is no diversity requirement for graduating from the incubator. In many ways, incubation hinders community growth. The requirement is that the project makes decisions as an Apache project, not in private, which is harder to get used to doing if a lot of people share the same office. Diversity is only a warning sign that means we need to check for decisions made in our forums and advise accordingly. It is not an end in itself, nor has lack of diversity hindered other projects from continuing on to build a larger community as a TLP. Roy On May 26, 2012, at 11:44 PM, Ralph Goersralph.go...@dslextreme.com wrote: On May 26, 2012, at 9:29 PM, Arvind Prabhakar wrote: Hi Jukka, On Sat, May 26, 2012 at 4:43 PM, Jukka Zittingjukka.zitt...@gmail.comwrote: IIUC Flume operates under an RTC model where people are not supposed to commit their own changes, which obviously makes the above data less relevant for evaluating the true diversity of the community. However, seeing only a single trivial commit by both jarcec and juhanic even though they became committers already over three months ago does seem to suggest that they may not be as comfortable in their committer role as people from Cloudera are. As you noted in your comments above - the Flume project tends to follow RTC with the reviewer committing the code. I happen to have taken up the role of the reviewer for the most part and hence you see the skewed commit counts. If you want to see the actual contribution, I would suggest looking at fixed JIRA issues by assignees. A quick report yields the following: aprabhkar - 26 - Cloudera [6] brocknoland - 19 - Cloudera [7] esammer - 56 - Cloudera [8] hshreedharan - 34 - Cloudera [9] jarcec - 6 - AVG Technologies [10] jmhsieh - 8 - Cloudera [11] juhanic - 9 - CyberAgent [12] mpercy - 34 - Cloudera [13] m...@apache.org - 1 - Trend Micro [14] prasadm - 34 - Cloudera [15] t...@cloudera.com - 3 - Cloudera [16] w...@cloudera.com - 3 - Cloudera [17] Looking at this, the average number of issues resolved by Cloudera committers (not counting Tom who is a mentor) is 26, and that for non-Cloudera committers is 5. Note that this number does not include other committer work such as the number of code reviews they have done, the number of design discussions they have participated in, something that is very valuable to the project. Another way of looking at these same statistics: Cloudera - 217 Other - 16 That means Cloudera is responsible for over 93% of the Jira issues. It is great that Cloudera is doing so much work but those stats hardly prove diversity. Ralph - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For
Re: Flume Graduation (was Re: June reports in two weeks)
On Tue, May 29, 2012 at 7:13 AM, Shane Curcuru a...@shanecurcuru.org wrote: Doesn't the IPMC (and not any individuals, even Roy) decide what the official Incubation policies should be? Ralph's reply, and my general reading of the feeling in the IPMC is that graduation votes *should* take affiliation diversity into account. Well, there's room for disagreement about the diversity requirement. That's why we have votes. Not that we *have* a diversity requirement, but rather how the situation at hand does or does not meet it. Note that some could argue Flume technically meets the diversity criteria, however I really like Jukka's analysis of who's actually making the commits - which means they may not necessarily meet the diversity criteria in spirit yet. - Shane P.S. For our public readers, note that Roy's emails are always worth reading. On 2012-05-27 5:17 AM, Ralph Goers wrote: Roy, What you are saying directly contradicts http://incubator.apache.org/guides/graduation.html#community, especially the statement Basically this means that when a project mostly consists of contributors from one company, this is a sign of not being diverse enough. Now, if that page is wrong and diversity is not a requirement for graduation then, of course, that would certainly remove any objections I have for Flume's graduation. However, I've heard it said that the ASF does not want to simply provide the infrastructure for companies to host their products on. Ralph On May 27, 2012, at 1:35 AM, Roy T. Fielding wrote: There is no diversity requirement for graduating from the incubator. In many ways, incubation hinders community growth. The requirement is that the project makes decisions as an Apache project, not in private, which is harder to get used to doing if a lot of people share the same office. Diversity is only a warning sign that means we need to check for decisions made in our forums and advise accordingly. It is not an end in itself, nor has lack of diversity hindered other projects from continuing on to build a larger community as a TLP. Roy On May 26, 2012, at 11:44 PM, Ralph Goersralph.go...@dslextreme.com wrote: On May 26, 2012, at 9:29 PM, Arvind Prabhakar wrote: Hi Jukka, On Sat, May 26, 2012 at 4:43 PM, Jukka Zittingjukka.zitt...@gmail.comwrote: IIUC Flume operates under an RTC model where people are not supposed to commit their own changes, which obviously makes the above data less relevant for evaluating the true diversity of the community. However, seeing only a single trivial commit by both jarcec and juhanic even though they became committers already over three months ago does seem to suggest that they may not be as comfortable in their committer role as people from Cloudera are. As you noted in your comments above - the Flume project tends to follow RTC with the reviewer committing the code. I happen to have taken up the role of the reviewer for the most part and hence you see the skewed commit counts. If you want to see the actual contribution, I would suggest looking at fixed JIRA issues by assignees. A quick report yields the following: aprabhkar - 26 - Cloudera [6] brocknoland - 19 - Cloudera [7] esammer - 56 - Cloudera [8] hshreedharan - 34 - Cloudera [9] jarcec - 6 - AVG Technologies [10] jmhsieh - 8 - Cloudera [11] juhanic - 9 - CyberAgent [12] mpercy - 34 - Cloudera [13] m...@apache.org - 1 - Trend Micro [14] prasadm - 34 - Cloudera [15] t...@cloudera.com - 3 - Cloudera [16] w...@cloudera.com - 3 - Cloudera [17] Looking at this, the average number of issues resolved by Cloudera committers (not counting Tom who is a mentor) is 26, and that for non-Cloudera committers is 5. Note that this number does not include other committer work such as the number of code reviews they have done, the number of design discussions they have participated in, something that is very valuable to the project. Another way of looking at these same statistics: Cloudera - 217 Other - 16 That means Cloudera is responsible for over 93% of the Jira issues. It is great that Cloudera is doing so much work but those stats hardly prove diversity. Ralph - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional
Re: Flume Graduation (was Re: June reports in two weeks)
Jukka Zitting wrote on Sun, May 27, 2012 at 11:40:36 +0200: It sounds like you have a reasonably good handle on that, so I'm not too worried, but my instinct suggests that the strict RTC model and distinction between committers and (P)PMC members may be structural factors that could easily end up tripping that balance. Are these really essential tools for the project or could you live without them? Other solutions to the RTC model include separate maintenance branches with stricter review and testing requirements, and the only cases where I really see a need for the committer/(P)PMC separation is with umbrella projects or special cases like GSoC students or co-operation across project boundaries. Subversion allows any committer (full or partial) to open an experimental branch at any time. This is comparable, I think, to encouraging CTR feature branches in an RTC community (with review at merge time --- like we did for some of stefan2's branches). http://subversion.apache.org/docs/community-guide/general#lightweight-branches - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
On Sun, May 27, 2012 at 12:21 PM, Arvind Prabhakar arv...@apache.org wrote: ...2. Regarding doing away with the difference between PPMC and committers, I am told that other projects do this during graduation. I.e., they promote all existing committers to PMC status during graduation... Not sure if that's a good idea - changing the PMC's dynamics (at least potentially, depending on how many committers are made PMC members) at the same time as the project is undergoing other changes due to graduation might be risky. If the project is willing to make all committers PMC members, I'd recommend doing that immediately, independently of graduation plans. -Bertrand - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
On 27 May 2012 00:43, Jukka Zitting jukka.zitt...@gmail.com wrote: arvind - 116 commits - Cloudera prasadm - 22 commits - Cloudera brock - 16 commits - Cloudera esammer - 4 commits - Cloudera jarcec - 1 commit - AVG Technologies juhanic - 1 commit - CyberAgent The only two non-Cloudera commits were pretty trivial changes (a plugin version upgrade [3] and a spelling fix [4]). My earlier classification of Flume as ready to graduate [5] was partially based on these new committers from outside Cloudera, but the above data suggests that they haven't been as active as I would have hoped. IIUC Flume operates under an RTC model where people are not supposed to commit their own changes, which obviously makes the above data less relevant for evaluating the true diversity of the community. However, seeing only a single trivial commit by both jarcec and juhanic even though they became committers already over three months ago does seem to suggest that they may not be as comfortable in their committer role as people from Cloudera are. Do you think this is a problem for the community? If yes, how do you plan to fix it? If not, why? I think RTC is a barrier to expansion/participation. I understand why it is viewed as critical in the core codebase of Hadoop -it's a key piece of code for the big web companies- but do you need RTC on any project in incubation? RTC creates a bias towards commits from organisations with 1 developer/committer, as its easier to ask a colleage over the desk to say review this than it is to stick something up and hope that it gets noticed by others.
Re: Flume Graduation (was Re: June reports in two weeks)
On 27 May 2012 01:15, Ralph Goers ralph.go...@dslextreme.com wrote: What would happen if all the Cloudera people were to suddenly vanish from the project (this could easily happen if Cloudera were purchased by a larger company who had different goals). As it stands today I don't believe the project would survive very long. That happened to axis 1.x when the IBM developers all got reassigned to Websphere stuff, a large chunk of the knowledge of the dev team vanished one day. That knowledge was missed more than the engineering support -the whole undercommented bit of the WSDL-to-Java-source metacode was the bit that was hardest for the successors to understand Sanjeeva and the WSO2 team took up the mantle, and with a focus on Axis made it much much better -showing that heavy (but not not sole) engagement from a small startup can benefit a project. Cloudera's biz plan does depend on a core OSS Hadoop, so it's less likely they will disappear than a team from some F100 corporation that can reassign or terminate whole organisations based on a single email. I'd put the risk more on in whose interests do decisions get made, rather than the they all disappear one day scenario. Even so, I hope the knowledge of the codebase is spread more widely than just across those who check stuff in.
Re: Flume Graduation (was Re: June reports in two weeks)
On May 26, 2012, at 9:29 PM, Arvind Prabhakar wrote: Hi Jukka, On Sat, May 26, 2012 at 4:43 PM, Jukka Zitting jukka.zitt...@gmail.comwrote: IIUC Flume operates under an RTC model where people are not supposed to commit their own changes, which obviously makes the above data less relevant for evaluating the true diversity of the community. However, seeing only a single trivial commit by both jarcec and juhanic even though they became committers already over three months ago does seem to suggest that they may not be as comfortable in their committer role as people from Cloudera are. As you noted in your comments above - the Flume project tends to follow RTC with the reviewer committing the code. I happen to have taken up the role of the reviewer for the most part and hence you see the skewed commit counts. If you want to see the actual contribution, I would suggest looking at fixed JIRA issues by assignees. A quick report yields the following: aprabhkar - 26 - Cloudera [6] brocknoland - 19 - Cloudera [7] esammer - 56 - Cloudera [8] hshreedharan - 34 - Cloudera [9] jarcec - 6 - AVG Technologies [10] jmhsieh - 8 - Cloudera [11] juhanic - 9 - CyberAgent [12] mpercy - 34 - Cloudera [13] m...@apache.org - 1 - Trend Micro [14] prasadm - 34 - Cloudera [15] t...@cloudera.com - 3 - Cloudera [16] w...@cloudera.com - 3 - Cloudera [17] Looking at this, the average number of issues resolved by Cloudera committers (not counting Tom who is a mentor) is 26, and that for non-Cloudera committers is 5. Note that this number does not include other committer work such as the number of code reviews they have done, the number of design discussions they have participated in, something that is very valuable to the project. Another way of looking at these same statistics: Cloudera - 217 Other - 16 That means Cloudera is responsible for over 93% of the Jira issues. It is great that Cloudera is doing so much work but those stats hardly prove diversity. Ralph - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
There is no diversity requirement for graduating from the incubator. In many ways, incubation hinders community growth. The requirement is that the project makes decisions as an Apache project, not in private, which is harder to get used to doing if a lot of people share the same office. Diversity is only a warning sign that means we need to check for decisions made in our forums and advise accordingly. It is not an end in itself, nor has lack of diversity hindered other projects from continuing on to build a larger community as a TLP. Roy On May 26, 2012, at 11:44 PM, Ralph Goers ralph.go...@dslextreme.com wrote: On May 26, 2012, at 9:29 PM, Arvind Prabhakar wrote: Hi Jukka, On Sat, May 26, 2012 at 4:43 PM, Jukka Zitting jukka.zitt...@gmail.comwrote: IIUC Flume operates under an RTC model where people are not supposed to commit their own changes, which obviously makes the above data less relevant for evaluating the true diversity of the community. However, seeing only a single trivial commit by both jarcec and juhanic even though they became committers already over three months ago does seem to suggest that they may not be as comfortable in their committer role as people from Cloudera are. As you noted in your comments above - the Flume project tends to follow RTC with the reviewer committing the code. I happen to have taken up the role of the reviewer for the most part and hence you see the skewed commit counts. If you want to see the actual contribution, I would suggest looking at fixed JIRA issues by assignees. A quick report yields the following: aprabhkar - 26 - Cloudera [6] brocknoland - 19 - Cloudera [7] esammer - 56 - Cloudera [8] hshreedharan - 34 - Cloudera [9] jarcec - 6 - AVG Technologies [10] jmhsieh - 8 - Cloudera [11] juhanic - 9 - CyberAgent [12] mpercy - 34 - Cloudera [13] m...@apache.org - 1 - Trend Micro [14] prasadm - 34 - Cloudera [15] t...@cloudera.com - 3 - Cloudera [16] w...@cloudera.com - 3 - Cloudera [17] Looking at this, the average number of issues resolved by Cloudera committers (not counting Tom who is a mentor) is 26, and that for non-Cloudera committers is 5. Note that this number does not include other committer work such as the number of code reviews they have done, the number of design discussions they have participated in, something that is very valuable to the project. Another way of looking at these same statistics: Cloudera - 217 Other - 16 That means Cloudera is responsible for over 93% of the Jira issues. It is great that Cloudera is doing so much work but those stats hardly prove diversity. Ralph - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
Roy, What you are saying directly contradicts http://incubator.apache.org/guides/graduation.html#community, especially the statement Basically this means that when a project mostly consists of contributors from one company, this is a sign of not being diverse enough. Now, if that page is wrong and diversity is not a requirement for graduation then, of course, that would certainly remove any objections I have for Flume's graduation. However, I've heard it said that the ASF does not want to simply provide the infrastructure for companies to host their products on. Ralph On May 27, 2012, at 1:35 AM, Roy T. Fielding wrote: There is no diversity requirement for graduating from the incubator. In many ways, incubation hinders community growth. The requirement is that the project makes decisions as an Apache project, not in private, which is harder to get used to doing if a lot of people share the same office. Diversity is only a warning sign that means we need to check for decisions made in our forums and advise accordingly. It is not an end in itself, nor has lack of diversity hindered other projects from continuing on to build a larger community as a TLP. Roy On May 26, 2012, at 11:44 PM, Ralph Goers ralph.go...@dslextreme.com wrote: On May 26, 2012, at 9:29 PM, Arvind Prabhakar wrote: Hi Jukka, On Sat, May 26, 2012 at 4:43 PM, Jukka Zitting jukka.zitt...@gmail.comwrote: IIUC Flume operates under an RTC model where people are not supposed to commit their own changes, which obviously makes the above data less relevant for evaluating the true diversity of the community. However, seeing only a single trivial commit by both jarcec and juhanic even though they became committers already over three months ago does seem to suggest that they may not be as comfortable in their committer role as people from Cloudera are. As you noted in your comments above - the Flume project tends to follow RTC with the reviewer committing the code. I happen to have taken up the role of the reviewer for the most part and hence you see the skewed commit counts. If you want to see the actual contribution, I would suggest looking at fixed JIRA issues by assignees. A quick report yields the following: aprabhkar - 26 - Cloudera [6] brocknoland - 19 - Cloudera [7] esammer - 56 - Cloudera [8] hshreedharan - 34 - Cloudera [9] jarcec - 6 - AVG Technologies [10] jmhsieh - 8 - Cloudera [11] juhanic - 9 - CyberAgent [12] mpercy - 34 - Cloudera [13] m...@apache.org - 1 - Trend Micro [14] prasadm - 34 - Cloudera [15] t...@cloudera.com - 3 - Cloudera [16] w...@cloudera.com - 3 - Cloudera [17] Looking at this, the average number of issues resolved by Cloudera committers (not counting Tom who is a mentor) is 26, and that for non-Cloudera committers is 5. Note that this number does not include other committer work such as the number of code reviews they have done, the number of design discussions they have participated in, something that is very valuable to the project. Another way of looking at these same statistics: Cloudera - 217 Other - 16 That means Cloudera is responsible for over 93% of the Jira issues. It is great that Cloudera is doing so much work but those stats hardly prove diversity. Ralph - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
Hi, On Sun, May 27, 2012 at 6:29 AM, Arvind Prabhakar arv...@apache.org wrote: As you noted in your comments above - the Flume project tends to follow RTC with the reviewer committing the code. I happen to have taken up the role of the reviewer for the most part and hence you see the skewed commit counts. It might be useful if you for example expanded the How to Commit wiki page [1] with a better description of what a reviewer should take in to account when committing reviewed changes. Things that might be obvious to you and others who've been around for longer, but not necessarily for newcomers. The more people you have committing code, the less the project is dependent on the silent knowledge of any single contributor. On Sat, May 26, 2012 at 4:43 PM, Jukka Zitting jukka.zitt...@gmail.comwrote: Do you think this is a problem for the community? If yes, how do you plan to fix it? If not, why? I don't think this is a problem because while most Cloudera committers have the luxury of working on the project during regular working hours, others do that during their off hours. Hence the majority of contributions coming from Cloudera. OK, fair enough. Such a scenario exists or has existed also in other Apache projects like Jackrabbit that I'm most familiar with. It can be a tricky balance to maintain a level playing field in such cases, for example by making sure that all relevant project discussions happen out in the open and that some committers don't feel like junior partners with less ability to influence the project. It sounds like you have a reasonably good handle on that, so I'm not too worried, but my instinct suggests that the strict RTC model and distinction between committers and (P)PMC members may be structural factors that could easily end up tripping that balance. Are these really essential tools for the project or could you live without them? Other solutions to the RTC model include separate maintenance branches with stricter review and testing requirements, and the only cases where I really see a need for the committer/(P)PMC separation is with umbrella projects or special cases like GSoC students or co-operation across project boundaries. I know you have good arguments for the way things work in Flume now, and ultimately you're the ones to decide how you want to work. So take the above just as friendly suggestions for things you may want to consider. Whatever the outcome, it would be good for Flume to document such decisions and the rationale behind them as a set of project bylaws. This is what the bylaws section of the graduation resolution is about: RESOLVED, that the initial Apache $project PMC be and hereby is tasked with the creation of a set of bylaws intended to encourage open development and increased participation in the Apache $foo Project; and be it further [1] https://cwiki.apache.org/confluence/display/FLUME/How+to+Commit BR, Jukka Zitting - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
Thanks Jukka for your suggestions. 1. Regarding wiki - I have taken a note of that and will update it soon. 2. Regarding doing away with the difference between PPMC and committers, I am told that other projects do this during graduation. I.e., they promote all existing committers to PMC status during graduation. If we do that, the diversity concern will be addressed and we can then debate the bylaws once the PMC is formed. Thanks, Arvind Prabhakar On Sun, May 27, 2012 at 2:40 AM, Jukka Zitting jukka.zitt...@gmail.comwrote: Hi, On Sun, May 27, 2012 at 6:29 AM, Arvind Prabhakar arv...@apache.org wrote: As you noted in your comments above - the Flume project tends to follow RTC with the reviewer committing the code. I happen to have taken up the role of the reviewer for the most part and hence you see the skewed commit counts. It might be useful if you for example expanded the How to Commit wiki page [1] with a better description of what a reviewer should take in to account when committing reviewed changes. Things that might be obvious to you and others who've been around for longer, but not necessarily for newcomers. The more people you have committing code, the less the project is dependent on the silent knowledge of any single contributor. On Sat, May 26, 2012 at 4:43 PM, Jukka Zitting jukka.zitt...@gmail.com wrote: Do you think this is a problem for the community? If yes, how do you plan to fix it? If not, why? I don't think this is a problem because while most Cloudera committers have the luxury of working on the project during regular working hours, others do that during their off hours. Hence the majority of contributions coming from Cloudera. OK, fair enough. Such a scenario exists or has existed also in other Apache projects like Jackrabbit that I'm most familiar with. It can be a tricky balance to maintain a level playing field in such cases, for example by making sure that all relevant project discussions happen out in the open and that some committers don't feel like junior partners with less ability to influence the project. It sounds like you have a reasonably good handle on that, so I'm not too worried, but my instinct suggests that the strict RTC model and distinction between committers and (P)PMC members may be structural factors that could easily end up tripping that balance. Are these really essential tools for the project or could you live without them? Other solutions to the RTC model include separate maintenance branches with stricter review and testing requirements, and the only cases where I really see a need for the committer/(P)PMC separation is with umbrella projects or special cases like GSoC students or co-operation across project boundaries. I know you have good arguments for the way things work in Flume now, and ultimately you're the ones to decide how you want to work. So take the above just as friendly suggestions for things you may want to consider. Whatever the outcome, it would be good for Flume to document such decisions and the rationale behind them as a set of project bylaws. This is what the bylaws section of the graduation resolution is about: RESOLVED, that the initial Apache $project PMC be and hereby is tasked with the creation of a set of bylaws intended to encourage open development and increased participation in the Apache $foo Project; and be it further [1] https://cwiki.apache.org/confluence/display/FLUME/How+to+Commit BR, Jukka Zitting - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
On Fri, May 25, 2012 at 12:29 PM, Ralph Goers ralph.go...@dslextreme.comwrote: At this point my recommendations are: 1. Since the PPMC voted to separate being a committer and being a PMC member I would wait a couple of months and then add the new non-Cloudera committers to the PPMC if it is warranted. 2. Of course, add any new committers who have earned it. 3. Send an email to the entire PPMC asking them to confirm that they want to remain on the PMC after graduation. 4. Based on that we will know what the making of the post-graduation PMC will look like. 5. Raise the topic of graduation as a discussion item either on the PMC or dev list after the above 4 are completed. What purpose will these recommendations serve? * We have established that there is sufficient diversity in the committers list. * Without counting you, there are two other non-Cloudera PPMC who have expressed their commitment to the project. * The VOTE thread has overwhelmingly established what the community wants. * We, the active PPMC have demonstrated self-governance using accepted Apache practices and are committed to growing and diversifying the project further. If at this stage you insist on postponing the graduation, I am inclined to think that perhaps your own preferences outweigh that of the project and community. Please don't get me wrong - I am not accusing you of anything, just requesting you to please consider revoking your -1 vote. Arvind Prabhakar Ralph - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
Hi, On Sat, May 26, 2012 at 7:11 PM, Arvind Prabhakar arv...@apache.org wrote: * We have established that there is sufficient diversity in the committers list. I tend to look more at actual commit activity than the committers list when evaluating the diversity of a project. There are people on the Flume committers list who've never committed anything. Combining information from svn statistics [1] and committer affiliations [2] I get the following list of Flume commit activity so far this year: arvind - 116 commits - Cloudera prasadm - 22 commits - Cloudera brock - 16 commits - Cloudera esammer - 4 commits - Cloudera jarcec - 1 commit - AVG Technologies juhanic - 1 commit - CyberAgent The only two non-Cloudera commits were pretty trivial changes (a plugin version upgrade [3] and a spelling fix [4]). My earlier classification of Flume as ready to graduate [5] was partially based on these new committers from outside Cloudera, but the above data suggests that they haven't been as active as I would have hoped. IIUC Flume operates under an RTC model where people are not supposed to commit their own changes, which obviously makes the above data less relevant for evaluating the true diversity of the community. However, seeing only a single trivial commit by both jarcec and juhanic even though they became committers already over three months ago does seem to suggest that they may not be as comfortable in their committer role as people from Cloudera are. Do you think this is a problem for the community? If yes, how do you plan to fix it? If not, why? [1] http://svnsearch.org/svnsearch/repos/ASF/search?from=20120101to=20121231path=%2Fincubator%2Fflume [2] http://incubator.apache.org/flume/team-list.html [3] http://svn.apache.org/viewvc?view=revisionrevision=1305478 [4] http://svn.apache.org/viewvc?view=revisionrevision=1342066 [5] http://markmail.org/message/kq5ixay6z3erw3pc BR, Jukka Zitting - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
On May 26, 2012, at 10:11 AM, Arvind Prabhakar wrote: On Fri, May 25, 2012 at 12:29 PM, Ralph Goers ralph.go...@dslextreme.comwrote: At this point my recommendations are: 1. Since the PPMC voted to separate being a committer and being a PMC member I would wait a couple of months and then add the new non-Cloudera committers to the PPMC if it is warranted. 2. Of course, add any new committers who have earned it. 3. Send an email to the entire PPMC asking them to confirm that they want to remain on the PMC after graduation. 4. Based on that we will know what the making of the post-graduation PMC will look like. 5. Raise the topic of graduation as a discussion item either on the PMC or dev list after the above 4 are completed. What purpose will these recommendations serve? * We have established that there is sufficient diversity in the committers list. * Without counting you, there are two other non-Cloudera PPMC who have expressed their commitment to the project. * The VOTE thread has overwhelmingly established what the community wants. * We, the active PPMC have demonstrated self-governance using accepted Apache practices and are committed to growing and diversifying the project further. If at this stage you insist on postponing the graduation, I am inclined to think that perhaps your own preferences outweigh that of the project and community. Please don't get me wrong - I am not accusing you of anything, just requesting you to please consider revoking your -1 vote. Arvind - my preferences are my following through as a mentor for Flume and insuring the project's success. That is my duty as a member of the ASF and the obligation I took on as a member of the IPMC. One measure of that would be to consider what would happen if all the Cloudera people were to suddenly vanish from the project (this could easily happen if Cloudera were purchased by a larger company who had different goals). As it stands today I don't believe the project would survive very long. Ralph - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
Hi Jukka, On Sat, May 26, 2012 at 4:43 PM, Jukka Zitting jukka.zitt...@gmail.comwrote: Hi, On Sat, May 26, 2012 at 7:11 PM, Arvind Prabhakar arv...@apache.org wrote: * We have established that there is sufficient diversity in the committers list. I tend to look more at actual commit activity than the committers list when evaluating the diversity of a project. There are people on the Flume committers list who've never committed anything. Combining information from svn statistics [1] and committer affiliations [2] I get the following list of Flume commit activity so far this year: arvind - 116 commits - Cloudera prasadm - 22 commits - Cloudera brock - 16 commits - Cloudera esammer - 4 commits - Cloudera jarcec - 1 commit - AVG Technologies juhanic - 1 commit - CyberAgent The only two non-Cloudera commits were pretty trivial changes (a plugin version upgrade [3] and a spelling fix [4]). My earlier classification of Flume as ready to graduate [5] was partially based on these new committers from outside Cloudera, but the above data suggests that they haven't been as active as I would have hoped. IIUC Flume operates under an RTC model where people are not supposed to commit their own changes, which obviously makes the above data less relevant for evaluating the true diversity of the community. However, seeing only a single trivial commit by both jarcec and juhanic even though they became committers already over three months ago does seem to suggest that they may not be as comfortable in their committer role as people from Cloudera are. As you noted in your comments above - the Flume project tends to follow RTC with the reviewer committing the code. I happen to have taken up the role of the reviewer for the most part and hence you see the skewed commit counts. If you want to see the actual contribution, I would suggest looking at fixed JIRA issues by assignees. A quick report yields the following: aprabhkar - 26 - Cloudera [6] brocknoland - 19 - Cloudera [7] esammer - 56 - Cloudera [8] hshreedharan - 34 - Cloudera [9] jarcec - 6 - AVG Technologies [10] jmhsieh - 8 - Cloudera [11] juhanic - 9 - CyberAgent [12] mpercy - 34 - Cloudera [13] m...@apache.org - 1 - Trend Micro [14] prasadm - 34 - Cloudera [15] t...@cloudera.com - 3 - Cloudera [16] w...@cloudera.com - 3 - Cloudera [17] Looking at this, the average number of issues resolved by Cloudera committers (not counting Tom who is a mentor) is 26, and that for non-Cloudera committers is 5. Note that this number does not include other committer work such as the number of code reviews they have done, the number of design discussions they have participated in, something that is very valuable to the project. Do you think this is a problem for the community? If yes, how do you plan to fix it? If not, why? I don't think this is a problem because while most Cloudera committers have the luxury of working on the project during regular working hours, others do that during their off hours. Hence the majority of contributions coming from Cloudera. If all committers were to be non-Cloudera, I am sure the averages will be low and comparable to what the non-Cloudera contribution rate is at the moment. In other words, the project growth will definitely slow down, but the community will remain healthy. This is also indicated by the following statistics: User List: Messages in May: 171, April: 119 Number of Subscribers: 102 Dev List: Messages in May: 642, April 1024 Number of Subscribers: 245 [6] http://s.apache.org/2hi [7] http://s.apache.org/kw [8] http://s.apache.org/nl [9] http://s.apache.org/SGe [10] http://s.apache.org/tR [11] http://s.apache.org/UvH [12] http://s.apache.org/A8V [13] http://s.apache.org/f8M [14] http://s.apache.org/TIK [15] http://s.apache.org/wO [16] http://s.apache.org/Fxb [17] http://s.apache.org/rw Thanks, Arvind Prabhakar [1] http://svnsearch.org/svnsearch/repos/ASF/search?from=20120101to=20121231path=%2Fincubator%2Fflume [2] http://incubator.apache.org/flume/team-list.html [3] http://svn.apache.org/viewvc?view=revisionrevision=1305478 [4] http://svn.apache.org/viewvc?view=revisionrevision=1342066 [5] http://markmail.org/message/kq5ixay6z3erw3pc BR, Jukka Zitting - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
Hi, On Thu, May 24, 2012 at 12:06 PM, Dave Fisher dave2w...@comcast.net wrote: I started reading some of the Flume website and I think that when you go to the main Wiki page: https://cwiki.apache.org/confluence/display/FLUME/Index When you click on the Flume Cookbook the resource is at cloudera.org. http://archive.cloudera.com/cdh/3/flume/Cookbook/ This page lists flume-...@cloudera.org and is a file with a revision dated May 7, 2012. Thanks. These have been removed. You can make you own conclusions, but it looks like podling resources need to be migrated to the ASF. These were kept to facilitate the transitoin over to Flume 0.9.5 which never happened. We instead went straight to 1.0.0. Other than that all resources have been migrated. Thanks, Arvind Prabhakar Regards, Dave In any case - I'm not insisting that the way the project is run needs to change. I'm simply saying I cannot support graduation with the current makeup of the committers and PMC. I don't have a hard and fast ratio - gaining 10 new unaffiliated committers who don't do much isn't nearly as good as 2 or 3 who are very active. Ultimately the project needs to figure out how to solve this. Stating that some committers who don't do much isn't nearly as good as 2 or 3 who are very active is an unfair characterization. This is more unfair for those who are part of the project but have not been active lately due to whatever reasons, but have played a foundational role in getting the project to a point where it is today. I think they are as important as any other committer who may be very active at the moment. Merit once earned, never expires [1]. [1] http://www.apache.org/dev/committers.html#committer-set-term I think you misunderstood my point or I didn't state it very well. Diversity isn't achieved simply by having bodies. IOW I am not suggesting offering commit rights to people who haven't earned it just to meet some ratio. However, I am not suggesting the project has ever even considered doing that. Ralph - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
On Thu, May 24, 2012 at 11:49 AM, Ralph Goers ralph.go...@dslextreme.comwrote: On May 24, 2012, at 10:40 AM, Arvind Prabhakar wrote: There are four companies represented in this list: AVG Technologies, Cloudera, CyberAgent and Trend Micro. Compared to other projects that have successfully graduated from Incubator in the past, this meets the diversity requirements very well. I was mistaken and the list above is indeed correct. For some reason I thought a couple of them had become Cloudera employees. However, none of those three are currently on the PPMC. When you look at the PPMC list you should also include a few more Cloudera people who do participate in release votes and PPMC issues. Most, if not all, of the non-Cloudera PMC members don't. Back in October of 2011 the PPMC discussed and decided to use the process where new committers are not automatically added to PPMC. We have since followed this process and added one committer to the PPMC recently. The current PPMC consists of: * Andrew Bayer (Cloudera) * Ahmed Radwan (Cloudera) * Arvind Prabhakar(Cloudera) * Bruce Mitchener (Independent) * Derek Deeter (Intuit) * Eric Sammer (Cloudera) * Henry Robinson (Cloudera) * Jonathan Hsieh (Cloudera) * Aaron Kimball (Odiago) * Ralph Goers (Intuit) There is a representation of at least four companies in this list. This, plus the the fact that we have demonstrated our commitment to the policies that we set forth, give me the confidence that we are on the right track. Diversity is a priority and will continue to be post graduation. Arvind In any case - I'm not insisting that the way the project is run needs to change. I'm simply saying I cannot support graduation with the current makeup of the committers and PMC. I don't have a hard and fast ratio - gaining 10 new unaffiliated committers who don't do much isn't nearly as good as 2 or 3 who are very active. Ultimately the project needs to figure out how to solve this. Stating that some committers who don't do much isn't nearly as good as 2 or 3 who are very active is an unfair characterization. This is more unfair for those who are part of the project but have not been active lately due to whatever reasons, but have played a foundational role in getting the project to a point where it is today. I think they are as important as any other committer who may be very active at the moment. Merit once earned, never expires [1]. [1] http://www.apache.org/dev/committers.html#committer-set-term I think you misunderstood my point or I didn't state it very well. Diversity isn't achieved simply by having bodies. IOW I am not suggesting offering commit rights to people who haven't earned it just to meet some ratio. However, I am not suggesting the project has ever even considered doing that. Ralph - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
On May 25, 2012, at 2:46 AM, Arvind Prabhakar wrote: On Thu, May 24, 2012 at 11:49 AM, Ralph Goers ralph.go...@dslextreme.comwrote: On May 24, 2012, at 10:40 AM, Arvind Prabhakar wrote: There are four companies represented in this list: AVG Technologies, Cloudera, CyberAgent and Trend Micro. Compared to other projects that have successfully graduated from Incubator in the past, this meets the diversity requirements very well. I was mistaken and the list above is indeed correct. For some reason I thought a couple of them had become Cloudera employees. However, none of those three are currently on the PPMC. When you look at the PPMC list you should also include a few more Cloudera people who do participate in release votes and PPMC issues. Most, if not all, of the non-Cloudera PMC members don't. Back in October of 2011 the PPMC discussed and decided to use the process where new committers are not automatically added to PPMC. We have since followed this process and added one committer to the PPMC recently. The current PPMC consists of: * Andrew Bayer (Cloudera) * Ahmed Radwan (Cloudera) * Arvind Prabhakar(Cloudera) * Bruce Mitchener (Independent) * Derek Deeter (Intuit) * Eric Sammer (Cloudera) * Henry Robinson (Cloudera) * Jonathan Hsieh (Cloudera) * Aaron Kimball (Odiago) * Ralph Goers (Intuit) There is a representation of at least four companies in this list. This, plus the the fact that we have demonstrated our commitment to the policies that we set forth, give me the confidence that we are on the right track. Diversity is a priority and will continue to be post graduation. Are you really going to resort to distortion of the truth to try to claim diversity? I work with Derek and know that he has never participated in Flume since its inception. I also can't recall Bruce or Aaron participating after the podling started. All of them appear to have just signed the initial wiki page to get on the PPMC. I'm not sure why you left Tom White and Patrick Hunt off as they are both mentors and are PPMC members and both have participated in PMC discussions. That leaves just me as the only non-Cloudera PPMC member who actively participates and I don't commit code and I've been on the PPMC primarily as a mentor. If you somehow believe that this constitutes diversity than my job as a mentor has a long way to go. Ralph - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
I was involved with Flume prior to coming to Apache ... then a baby arrived and things sort of changed for the last year. But you're right that I haven't been around since then ... I still have a couple of source trees around with various changes in them, but no idea what the status of any of that is. :/ - Bruce On Fri, May 25, 2012 at 10:03 PM, Ralph Goers ralph.go...@dslextreme.comwrote: Are you really going to resort to distortion of the truth to try to claim diversity? I work with Derek and know that he has never participated in Flume since its inception. I also can't recall Bruce or Aaron participating after the podling started. All of them appear to have just signed the initial wiki page to get on the PPMC.
Re: Flume Graduation (was Re: June reports in two weeks)
On 24 May 2012 07:44, Marvin Humphrey mar...@rectangular.com wrote: To me that seems like it raises a barrier to entry -- but then, there are numerous projects around the ASF who are not hurting for contributors and who use JIRA for *everything* -- starting with Hadoop and Lucene. I first encountered when I subbed to the Maven dist for a while. Very different from ant-dev, which was pure distributed community and axis-dev, which had some commercial team engagement (IBM), but in which WSO2 strive hard to not be exclusionary. -- Marvin Humphrey, who in moments of weakness fantasizes about the day when Infra can no longer keep the massive ASF JIRA instance from toppling over and all the Java projects whose participation in Infra is limited to complaining when stuff goes offline come crying. It is becoming a bit of a SPOF, isn't it?
Re: Flume Graduation (was Re: June reports in two weeks)
On 24 May 2012 06:15, Benson Margulies bimargul...@gmail.com wrote: I've met other groups of people who like a JIRA centric view of the world. I suspect that if they did a bunch of other good things called out below, you or others would find the JIRA business digestible. Also, on the other hand, I fear that the co-employed contributors are collaborating in the hallway, and the lack of the context in JIRA or on the list is contributing to the problem. I'm not convinced that JIRA helps communities. It's great in companies -IDE integration, you can bounce issues to others, it pings your phone so often you can use it as a network liveness test. It also lets you persist discussions in a way that can be searched. In a busy project, it helps you keep track of your workload, and can assist in sprint planning if you fill in the est/actual workload fields. But -it encourages people to watch the issues they care about, and ignore the rest -it pushes you towards discussion-on-jira rather than in the mailing list -those discussions tend to be focused, rather than generic community chitchat about dev issues. When you compare it to the community of the pure-email list groups, or the socialness of github, JIRA seems to me that it pushes you towards antisocialism, to sit in your office and care only about the 8 JIRAs you have marked as in progress. Given the ubiquity and value of JIRA, what can be done here?
Re: Flume Graduation (was Re: June reports in two weeks)
On Fri, May 25, 2012 at 8:03 AM, Ralph Goers ralph.go...@dslextreme.comwrote: On May 25, 2012, at 2:46 AM, Arvind Prabhakar wrote: On Thu, May 24, 2012 at 11:49 AM, Ralph Goers ralph.go...@dslextreme.comwrote: On May 24, 2012, at 10:40 AM, Arvind Prabhakar wrote: The current PPMC consists of: * Andrew Bayer (Cloudera) * Ahmed Radwan (Cloudera) * Arvind Prabhakar(Cloudera) * Bruce Mitchener (Independent) * Derek Deeter (Intuit) * Eric Sammer (Cloudera) * Henry Robinson (Cloudera) * Jonathan Hsieh (Cloudera) * Aaron Kimball (Odiago) * Ralph Goers (Intuit) There is a representation of at least four companies in this list. This, plus the the fact that we have demonstrated our commitment to the policies that we set forth, give me the confidence that we are on the right track. Diversity is a priority and will continue to be post graduation. Are you really going to resort to distortion of the truth to try to claim diversity? I work with Derek and know that he has never participated in Flume since its inception. I also can't recall Bruce or Aaron participating after the podling started. All of them appear to have just signed the initial wiki page to get on the PPMC. It is unfortunate that you think this way. Bruce and Aaron have played a foundational role in Flume and they deserve to be seated on the PPMC unless they explicitly request to be removed. I welcome their contribution anytime and hope that they chose to remain on the project. I'm not sure why you left Tom White and Patrick Hunt off as they are both mentors and are PPMC members and both have participated in PMC discussions. Because I counted them as mentors and not PPMC. I am happy to include them if they want to be counted as PPMC. Pardon me if I should not have made this distinction. The one person who was left out inadvertently was Prasad who recently got elected to PPMC. That leaves just me as the only non-Cloudera PPMC member who actively participates and I don't commit code and I've been on the PPMC primarily as a mentor. If you somehow believe that this constitutes diversity than my job as a mentor has a long way to go. You were counted because three months ago you announced that you would like to stay with the project post graduation. I don't remember seeing any communication from you stating your change of mind. That said, I respect your choice either way. Ralph - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
On May 25, 2012, at 11:03 AM, Arvind Prabhakar wrote: That leaves just me as the only non-Cloudera PPMC member who actively participates and I don't commit code and I've been on the PPMC primarily as a mentor. If you somehow believe that this constitutes diversity than my job as a mentor has a long way to go. You were counted because three months ago you announced that you would like to stay with the project post graduation. I don't remember seeing any communication from you stating your change of mind. That said, I respect your choice either way. Arvind, I don't take disagreements like this personally and I hope you don't either. I have every intention of staying with the project after graduation and hopefully, finding a way to commit code as there are a number of areas I believe I can contribute. At this point my recommendations are: 1. Since the PPMC voted to separate being a committer and being a PMC member I would wait a couple of months and then add the new non-Cloudera committers to the PPMC if it is warranted. 2. Of course, add any new committers who have earned it. 3. Send an email to the entire PPMC asking them to confirm that they want to remain on the PMC after graduation. 4. Based on that we will know what the making of the post-graduation PMC will look like. 5. Raise the topic of graduation as a discussion item either on the PMC or dev list after the above 4 are completed. Ralph - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
Hi folks, Just to throw my hat into the ring on this subject: I recognize that I have been inactive from a contribution standpoint since Flume's introduction into the ASF incubator, but still follow Flume on the mailing lists and in other community meetups. I've done a bunch of work with Flume in the past, and I remain very interested in where Flume is going. Flume's direction is something important to me; but I have seldom weighed in, as I often felt that others were resolving things in a manner that did not require my input. Unfortunately, my duties with my own startup have prevented me from taking a more direct role in the building of Flume for quite some time. I believe that there are ways in which it makes sense for WibiData to directly integrate with Flume in the future -- but time and resources are scarce at a startup, so this has not been something I've yet demonstrated externally in front of the incubator community. I look forward to a time in the future when my duties are rearranged once again, and I can devote more of my time back to hacking than I do now :) (The one constant at a startup, after all, is change.) Regards, - Aaron Kimball PS -- My affiliation should be updated to WibiData; we changed the company name from Odiago to WibiData recently. On Fri, May 25, 2012 at 12:29 PM, Ralph Goers ralph.go...@dslextreme.comwrote: On May 25, 2012, at 11:03 AM, Arvind Prabhakar wrote: That leaves just me as the only non-Cloudera PPMC member who actively participates and I don't commit code and I've been on the PPMC primarily as a mentor. If you somehow believe that this constitutes diversity than my job as a mentor has a long way to go. You were counted because three months ago you announced that you would like to stay with the project post graduation. I don't remember seeing any communication from you stating your change of mind. That said, I respect your choice either way. Arvind, I don't take disagreements like this personally and I hope you don't either. I have every intention of staying with the project after graduation and hopefully, finding a way to commit code as there are a number of areas I believe I can contribute. At this point my recommendations are: 1. Since the PPMC voted to separate being a committer and being a PMC member I would wait a couple of months and then add the new non-Cloudera committers to the PPMC if it is warranted. 2. Of course, add any new committers who have earned it. 3. Send an email to the entire PPMC asking them to confirm that they want to remain on the PMC after graduation. 4. Based on that we will know what the making of the post-graduation PMC will look like. 5. Raise the topic of graduation as a discussion item either on the PMC or dev list after the above 4 are completed. Ralph - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
On May 23, 2012, at 10:48 PM, Patrick Hunt wrote: On Wed, May 23, 2012 at 10:36 PM, Ralph Goers ralph.go...@dslextreme.com wrote: On May 23, 2012, at 10:15 PM, Benson Margulies wrote: On Wed, May 23, 2012 at 10:09 PM, Ralph Goers ralph.go...@dslextreme.com wrote: Right after I read Jukka's email that started this thread and I posted my reply and discovered to my shock that they had started a graduation vote. I am shocked because I have pointed out repeatedly the project's complete lack of diversity. Virtually all the active PMC members and committers work for the same employer. I have told them several times that I would actually like to participate in the project but the way the project works is very different then every other project I am involved with at the ASF and the barriers to figure out what is actually going on is very high. Almost nothing is discussed directly on the dev list - it is all done through Jira issues or the Review tool. While all the Jira issue updates and reviews are sent to the dev list most of that is just noise. Feel free to review the dev list archives to see what I am talking about. I don't follow flume, but I'd propose to soften your objection only slightly. I've met other groups of people who like a JIRA centric view of the world. I suspect that if they did a bunch of other good things called out below, you or others would find the JIRA business digestible. Also, on the other hand, I fear that the co-employed contributors are collaborating in the hallway, and the lack of the context in JIRA or on the list is contributing to the problem. I have reason to doubt the collaboration in the hallway aspect and I certainly do not doubt everyone's good intent. I'm not objecting to the collaboration style as an issue preventing graduation. I'm just saying I find it difficult to participate with that style and that simply makes me wonder if that is making it harder to attract new committers. I fully realize that that issue might just be with me, but the fact remains that there is practically no diversity in the project and I cannot in good conscience recommend graduation for a project in that situation. Hi Ralph, Benson, et. al., some background: Flume is similar to Hadoop and other related projects in that it is very jira heavy for development activity. No slouch in terms of mailing list traffic either though (1200 last month): http://flume.markmail.org/ Also note the extensive new developer type detail that's available on the web/wiki: https://cwiki.apache.org/confluence/display/FLUME/Index The team list can provide insight into the diversity issue http://incubator.apache.org/flume/team-list.html My understanding is that there are at least 4 separate organizations represented by active commiters. The team list is incorrect and is somewhat misleading. To my knowledge at least two separate organizations represented in that list are now employed by Cloudera. Others signed on when the project entered the incubator but have never participated. This all became clear to me during the last release vote when, as I recall, I cast the only binding vote that didn't come from a Cloudera employee. Ralph Regards, Patrick Needless to say, when the graduation proposal reaches this list, and I'm sure it will, I will strongly endorse the IPMC to reject the proposal. FWIW, I found the post below to be 100% on target. Ralph On May 23, 2012, at 7:31 PM, Marvin Humphrey wrote: On Wed, May 23, 2012 at 5:36 PM, Patrick Hunt ph...@apache.org wrote: Perhaps someone will have some insight on how to gather new contributors that hasn't been tried yet? Jukka's written on this subject multiple times in the past. Here are two gems, one from a while back, the other recent: http://markmail.org/message/o3gbgam4ny2upqte Most of the cases I've been involved so far of podlings in the hoping some more people come along have had symptoms of the project team not paying enough attention on making it easy for new contributors to show up and stick around. Things like complex and undocumented build steps, missing Getting started or Getting involved guides, lack of quick and positive feedback to newcomers, etc., are all too common. Fixing even just some of such things will dramatically increase the odds of new people showing up. Those are things that are very easy to overlook when you're working on your first open source projects (it took me years to learn those lessons), but we here have a massive amount of collective experience on such things. That's what we could and IMHO should be sharing with the podlings. That's what mentoring to me is about and that's where our most precious added value is. Otherwise incubation just boils down to an indoctrination period on how to apply and conform to the various Apache
Re: Flume Graduation (was Re: June reports in two weeks)
On May 23, 2012, at 10:48 PM, Patrick Hunt wrote: On Wed, May 23, 2012 at 10:36 PM, Ralph Goers ralph.go...@dslextreme.com wrote: On May 23, 2012, at 10:15 PM, Benson Margulies wrote: On Wed, May 23, 2012 at 10:09 PM, Ralph Goers ralph.go...@dslextreme.com wrote: Right after I read Jukka's email that started this thread and I posted my reply and discovered to my shock that they had started a graduation vote. I am shocked because I have pointed out repeatedly the project's complete lack of diversity. Virtually all the active PMC members and committers work for the same employer. I have told them several times that I would actually like to participate in the project but the way the project works is very different then every other project I am involved with at the ASF and the barriers to figure out what is actually going on is very high. Almost nothing is discussed directly on the dev list - it is all done through Jira issues or the Review tool. While all the Jira issue updates and reviews are sent to the dev list most of that is just noise. Feel free to review the dev list archives to see what I am talking about. I don't follow flume, but I'd propose to soften your objection only slightly. I've met other groups of people who like a JIRA centric view of the world. I suspect that if they did a bunch of other good things called out below, you or others would find the JIRA business digestible. Also, on the other hand, I fear that the co-employed contributors are collaborating in the hallway, and the lack of the context in JIRA or on the list is contributing to the problem. I have reason to doubt the collaboration in the hallway aspect and I certainly do not doubt everyone's good intent. I'm not objecting to the collaboration style as an issue preventing graduation. I'm just saying I find it difficult to participate with that style and that simply makes me wonder if that is making it harder to attract new committers. I fully realize that that issue might just be with me, but the fact remains that there is practically no diversity in the project and I cannot in good conscience recommend graduation for a project in that situation. Hi Ralph, Benson, et. al., some background: Flume is similar to Hadoop and other related projects in that it is very jira heavy for development activity. No slouch in terms of mailing list traffic either though (1200 last month): http://flume.markmail.org/ Sorry I didn't include this in my prior post but here you are making my point exactly. I participate in several other Apache projects. Wading through 1200+ emails per month that are largely Jira/Review noise makes it very difficult for me to find posts that have any value. As a consequence I am largely forced to simply delete everything generated by he Review tool and Jira. And I'm a mentor. I just don't see how newcomers are going to find this style welcoming. Ralph - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
On Wed, May 23, 2012 at 11:18 PM, Ralph Goers ralph.go...@dslextreme.com wrote: On May 23, 2012, at 10:48 PM, Patrick Hunt wrote: On Wed, May 23, 2012 at 10:36 PM, Ralph Goers ralph.go...@dslextreme.com wrote: On May 23, 2012, at 10:15 PM, Benson Margulies wrote: On Wed, May 23, 2012 at 10:09 PM, Ralph Goers ralph.go...@dslextreme.com wrote: Right after I read Jukka's email that started this thread and I posted my reply and discovered to my shock that they had started a graduation vote. I am shocked because I have pointed out repeatedly the project's complete lack of diversity. Virtually all the active PMC members and committers work for the same employer. I have told them several times that I would actually like to participate in the project but the way the project works is very different then every other project I am involved with at the ASF and the barriers to figure out what is actually going on is very high. Almost nothing is discussed directly on the dev list - it is all done through Jira issues or the Review tool. While all the Jira issue updates and reviews are sent to the dev list most of that is just noise. Feel free to review the dev list archives to see what I am talking about. I don't follow flume, but I'd propose to soften your objection only slightly. I've met other groups of people who like a JIRA centric view of the world. I suspect that if they did a bunch of other good things called out below, you or others would find the JIRA business digestible. Also, on the other hand, I fear that the co-employed contributors are collaborating in the hallway, and the lack of the context in JIRA or on the list is contributing to the problem. I have reason to doubt the collaboration in the hallway aspect and I certainly do not doubt everyone's good intent. I'm not objecting to the collaboration style as an issue preventing graduation. I'm just saying I find it difficult to participate with that style and that simply makes me wonder if that is making it harder to attract new committers. I fully realize that that issue might just be with me, but the fact remains that there is practically no diversity in the project and I cannot in good conscience recommend graduation for a project in that situation. Hi Ralph, Benson, et. al., some background: Flume is similar to Hadoop and other related projects in that it is very jira heavy for development activity. No slouch in terms of mailing list traffic either though (1200 last month): http://flume.markmail.org/ Sorry I didn't include this in my prior post but here you are making my point exactly. I participate in several other Apache projects. Wading through 1200+ emails per month that are largely Jira/Review noise makes it very difficult for me to find posts that have any value. As a consequence I am largely forced to simply delete everything generated by he Review tool and Jira. And I'm a mentor. I just don't see how newcomers are going to find this style welcoming. There are separate lists it's just that markmail clubs them all together. It's also pretty easy to filter... Patrick - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
On Wed, May 23, 2012 at 10:36 PM, Ralph Goers ralph.go...@dslextreme.com wrote: I'm just saying I find it difficult to participate with that style and that simply makes me wonder if that is making it harder to attract new committers. I suspect it attracts some and drives away others. I frikkin' hate JIRA notifications. The emails suck, so newcomers are forced to learn JIRA's interface before they can participate fully in the dev conversation. To me that seems like it raises a barrier to entry -- but then, there are numerous projects around the ASF who are not hurting for contributors and who use JIRA for *everything* -- starting with Hadoop and Lucene. If you don't want JIRA-centric development, it can be curtailed by sending notifications to a dedicated issues list instead of the dev list. However, I would not necessarily recommend that to a new podling, as I can't tell where my own biases end and I don't want to start a phpBB^H^H^H^H^HJIRA vs email flame war. -- Marvin Humphrey, who in moments of weakness fantasizes about the day when Infra can no longer keep the massive ASF JIRA instance from toppling over and all the Java projects whose participation in Infra is limited to complaining when stuff goes offline come crying. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
I appreciate your position Ralph and I don't want anyone to feel like they can't contribute. As we've talked about before, we've been quick to nurture new contributors to committer status successfully in a few cases. It's true that some of the more active committers are from Cloudera, but it's not to the exclusion of anyone. Others aren't from Cloudera. Those of us that work together are also very strict about abiding to the if it's not on the mailing list, it didn't happen rule (where mailing list can mean JIRA or other ASF infrastructure as well). I'm happy to take your guidance as a mentor, but you also need to understand that some of the ways the Flume project has elected to operate are just a matter of taste. They were proposed, discussed, voted on (and not as a block by Cloudera employees, IIRC - pretty sure I was -0), and put in place and do not violate the Apache Way (like RTC vs. CTR). They aren't unheard of and they do not work to the exclusion of contributors (RTC, for instance, only impacts committers). I think the vote that was started was only to gauge community opinion as a first step (although I'm not completely well versed in the graduation process, to be honest). If there are concrete things we can do to improve diversity, in your opinion, I am extremely open to hearing them. We already do many of the (excellent) things listed earlier in the thread. JIRA noise withstanding (again, it's a matter of taste - I use the email frequently as I find trolling through JIRA slow) I'm definitely open to ideas. Of course, if Flume simply needs to remain in the incubator until we develop greater diversity, that's fine too. If we're not ready, we're just not ready. On Wed, May 23, 2012 at 11:18 PM, Ralph Goers ralph.go...@dslextreme.comwrote: On May 23, 2012, at 10:48 PM, Patrick Hunt wrote: On Wed, May 23, 2012 at 10:36 PM, Ralph Goers ralph.go...@dslextreme.com wrote: On May 23, 2012, at 10:15 PM, Benson Margulies wrote: On Wed, May 23, 2012 at 10:09 PM, Ralph Goers ralph.go...@dslextreme.com wrote: Right after I read Jukka's email that started this thread and I posted my reply and discovered to my shock that they had started a graduation vote. I am shocked because I have pointed out repeatedly the project's complete lack of diversity. Virtually all the active PMC members and committers work for the same employer. I have told them several times that I would actually like to participate in the project but the way the project works is very different then every other project I am involved with at the ASF and the barriers to figure out what is actually going on is very high. Almost nothing is discussed directly on the dev list - it is all done through Jira issues or the Review tool. While all the Jira issue updates and reviews are sent to the dev list most of that is just noise. Feel free to review the dev list archives to see what I am talking about. I don't follow flume, but I'd propose to soften your objection only slightly. I've met other groups of people who like a JIRA centric view of the world. I suspect that if they did a bunch of other good things called out below, you or others would find the JIRA business digestible. Also, on the other hand, I fear that the co-employed contributors are collaborating in the hallway, and the lack of the context in JIRA or on the list is contributing to the problem. I have reason to doubt the collaboration in the hallway aspect and I certainly do not doubt everyone's good intent. I'm not objecting to the collaboration style as an issue preventing graduation. I'm just saying I find it difficult to participate with that style and that simply makes me wonder if that is making it harder to attract new committers. I fully realize that that issue might just be with me, but the fact remains that there is practically no diversity in the project and I cannot in good conscience recommend graduation for a project in that situation. Hi Ralph, Benson, et. al., some background: Flume is similar to Hadoop and other related projects in that it is very jira heavy for development activity. No slouch in terms of mailing list traffic either though (1200 last month): http://flume.markmail.org/ Sorry I didn't include this in my prior post but here you are making my point exactly. I participate in several other Apache projects. Wading through 1200+ emails per month that are largely Jira/Review noise makes it very difficult for me to find posts that have any value. As a consequence I am largely forced to simply delete everything generated by he Review tool and Jira. And I'm a mentor. I just don't see how newcomers are going to find this style welcoming. Ralph - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org -- Eric Sammer twitter:
Re: Flume Graduation (was Re: June reports in two weeks)
The ONLY issue I see for Flume to graduate is diversity. No one will convince me that the current makeup constitutes diversity of any kind. Perhaps I shouldn't have brought up the mailing list issues as that was only meant in the spirit of trying to offer some advice on how more diversity could be achieved. Flume is really the only community I participate in that contains Cloudera employees so I do find myself wondering if the way the project is run is because that is the way all projects with a large number of Cloudera employees are run. That might make all of those participants comfortable but might create a barrier to others. In any case - I'm not insisting that the way the project is run needs to change. I'm simply saying I cannot support graduation with the current makeup of the committers and PMC. I don't have a hard and fast ratio - gaining 10 new unaffiliated committers who don't do much isn't nearly as good as 2 or 3 who are very active. Ultimately the project needs to figure out how to solve this. Ralph On May 23, 2012, at 11:48 PM, Eric Sammer wrote: I appreciate your position Ralph and I don't want anyone to feel like they can't contribute. As we've talked about before, we've been quick to nurture new contributors to committer status successfully in a few cases. It's true that some of the more active committers are from Cloudera, but it's not to the exclusion of anyone. Others aren't from Cloudera. Those of us that work together are also very strict about abiding to the if it's not on the mailing list, it didn't happen rule (where mailing list can mean JIRA or other ASF infrastructure as well). I'm happy to take your guidance as a mentor, but you also need to understand that some of the ways the Flume project has elected to operate are just a matter of taste. They were proposed, discussed, voted on (and not as a block by Cloudera employees, IIRC - pretty sure I was -0), and put in place and do not violate the Apache Way (like RTC vs. CTR). They aren't unheard of and they do not work to the exclusion of contributors (RTC, for instance, only impacts committers). I think the vote that was started was only to gauge community opinion as a first step (although I'm not completely well versed in the graduation process, to be honest). If there are concrete things we can do to improve diversity, in your opinion, I am extremely open to hearing them. We already do many of the (excellent) things listed earlier in the thread. JIRA noise withstanding (again, it's a matter of taste - I use the email frequently as I find trolling through JIRA slow) I'm definitely open to ideas. Of course, if Flume simply needs to remain in the incubator until we develop greater diversity, that's fine too. If we're not ready, we're just not ready. On Wed, May 23, 2012 at 11:18 PM, Ralph Goers ralph.go...@dslextreme.comwrote: On May 23, 2012, at 10:48 PM, Patrick Hunt wrote: On Wed, May 23, 2012 at 10:36 PM, Ralph Goers ralph.go...@dslextreme.com wrote: On May 23, 2012, at 10:15 PM, Benson Margulies wrote: On Wed, May 23, 2012 at 10:09 PM, Ralph Goers ralph.go...@dslextreme.com wrote: Right after I read Jukka's email that started this thread and I posted my reply and discovered to my shock that they had started a graduation vote. I am shocked because I have pointed out repeatedly the project's complete lack of diversity. Virtually all the active PMC members and committers work for the same employer. I have told them several times that I would actually like to participate in the project but the way the project works is very different then every other project I am involved with at the ASF and the barriers to figure out what is actually going on is very high. Almost nothing is discussed directly on the dev list - it is all done through Jira issues or the Review tool. While all the Jira issue updates and reviews are sent to the dev list most of that is just noise. Feel free to review the dev list archives to see what I am talking about. I don't follow flume, but I'd propose to soften your objection only slightly. I've met other groups of people who like a JIRA centric view of the world. I suspect that if they did a bunch of other good things called out below, you or others would find the JIRA business digestible. Also, on the other hand, I fear that the co-employed contributors are collaborating in the hallway, and the lack of the context in JIRA or on the list is contributing to the problem. I have reason to doubt the collaboration in the hallway aspect and I certainly do not doubt everyone's good intent. I'm not objecting to the collaboration style as an issue preventing graduation. I'm just saying I find it difficult to participate with that style and that simply makes me wonder if that is making it harder to attract new committers. I fully realize that that issue might just be with me, but the fact remains that there is
Re: Flume Graduation (was Re: June reports in two weeks)
On May 24, 2012, at 12:20 AM, Ralph Goers ralph.go...@dslextreme.com wrote: The ONLY issue I see for Flume to graduate is diversity. No one will convince me that the current makeup constitutes diversity of any kind. Perhaps I shouldn't have brought up the mailing list issues as that was only meant in the spirit of trying to offer some advice on how more diversity could be achieved. Flume is really the only community I participate in that contains Cloudera employees so I do find myself wondering if the way the project is run is because that is the way all projects with a large number of Cloudera employees are run. That might make all of those participants comfortable but might create a barrier to others. There are others where this is the case that are easily referenceable. There's an obvious (to me) implication that this is the cause of the problem and that's simply not true. If there are concrete recommendations of things you feel we can do better I know the flume community is open to those sightings. There's no practice in place within flume that isn't in place in some other ASF TLP to my knowledge. In any case - I'm not insisting that the way the project is run needs to change. I'm simply saying I cannot support graduation with the current makeup of the committers and PMC. I don't have a hard and fast ratio - gaining 10 new unaffiliated committers who don't do much isn't nearly as good as 2 or 3 who are very active. Ultimately the project needs to figure out how to solve this. That's fine. So let's have a discussion about actionable tasks. I've mentioned my thoughts on growing diversity in the past, although admittedly it was within a response to a similar thread on our private list. I'll start a thread on our dev list with the same thoughts for the larger community to comment on. I welcome your contribution to such a discussion! Thanks. Ralph On May 23, 2012, at 11:48 PM, Eric Sammer wrote: I appreciate your position Ralph and I don't want anyone to feel like they can't contribute. As we've talked about before, we've been quick to nurture new contributors to committer status successfully in a few cases. It's true that some of the more active committers are from Cloudera, but it's not to the exclusion of anyone. Others aren't from Cloudera. Those of us that work together are also very strict about abiding to the if it's not on the mailing list, it didn't happen rule (where mailing list can mean JIRA or other ASF infrastructure as well). I'm happy to take your guidance as a mentor, but you also need to understand that some of the ways the Flume project has elected to operate are just a matter of taste. They were proposed, discussed, voted on (and not as a block by Cloudera employees, IIRC - pretty sure I was -0), and put in place and do not violate the Apache Way (like RTC vs. CTR). They aren't unheard of and they do not work to the exclusion of contributors (RTC, for instance, only impacts committers). I think the vote that was started was only to gauge community opinion as a first step (although I'm not completely well versed in the graduation process, to be honest). If there are concrete things we can do to improve diversity, in your opinion, I am extremely open to hearing them. We already do many of the (excellent) things listed earlier in the thread. JIRA noise withstanding (again, it's a matter of taste - I use the email frequently as I find trolling through JIRA slow) I'm definitely open to ideas. Of course, if Flume simply needs to remain in the incubator until we develop greater diversity, that's fine too. If we're not ready, we're just not ready. On Wed, May 23, 2012 at 11:18 PM, Ralph Goers ralph.go...@dslextreme.comwrote: On May 23, 2012, at 10:48 PM, Patrick Hunt wrote: On Wed, May 23, 2012 at 10:36 PM, Ralph Goers ralph.go...@dslextreme.com wrote: On May 23, 2012, at 10:15 PM, Benson Margulies wrote: On Wed, May 23, 2012 at 10:09 PM, Ralph Goers ralph.go...@dslextreme.com wrote: Right after I read Jukka's email that started this thread and I posted my reply and discovered to my shock that they had started a graduation vote. I am shocked because I have pointed out repeatedly the project's complete lack of diversity. Virtually all the active PMC members and committers work for the same employer. I have told them several times that I would actually like to participate in the project but the way the project works is very different then every other project I am involved with at the ASF and the barriers to figure out what is actually going on is very high. Almost nothing is discussed directly on the dev list - it is all done through Jira issues or the Review tool. While all the Jira issue updates and reviews are sent to the dev list most of that is just noise. Feel free to review the dev list archives to see what I am talking about. I don't follow flume, but I'd propose to soften your
Re: Flume Graduation (was Re: June reports in two weeks)
Hi, On Thu, May 24, 2012 at 12:19 AM, Ralph Goers ralph.go...@dslextreme.comwrote: The ONLY issue I see for Flume to graduate is diversity. No one will convince me that the current makeup constitutes diversity of any kind. Perhaps I shouldn't have brought up the mailing list issues as that was only meant in the spirit of trying to offer some advice on how more diversity could be achieved. Flume is really the only community I participate in that contains Cloudera employees so I do find myself wondering if the way the project is run is because that is the way all projects with a large number of Cloudera employees are run. That might make all of those participants comfortable but might create a barrier to others. Here are the committers who have been active in the past three months: * Brock Noland (Cloudera) * Hari Shreedharan (Cloudera) * Jarek Jarcec Cecho (AVG Technologies) * Juhani Connolly (CyberAgent) * Mike Percy (Cloudera) * Mingjie Lai (Trend Micro) * Prasad Mujumdar (Cloudera) * Will McQueen (Cloudera) * Arvind Prabhakar (Cloudera) There are four companies represented in this list: AVG Technologies, Cloudera, CyberAgent and Trend Micro. Compared to other projects that have successfully graduated from Incubator in the past, this meets the diversity requirements very well. In any case - I'm not insisting that the way the project is run needs to change. I'm simply saying I cannot support graduation with the current makeup of the committers and PMC. I don't have a hard and fast ratio - gaining 10 new unaffiliated committers who don't do much isn't nearly as good as 2 or 3 who are very active. Ultimately the project needs to figure out how to solve this. Stating that some committers who don't do much isn't nearly as good as 2 or 3 who are very active is an unfair characterization. This is more unfair for those who are part of the project but have not been active lately due to whatever reasons, but have played a foundational role in getting the project to a point where it is today. I think they are as important as any other committer who may be very active at the moment. Merit once earned, never expires [1]. [1] http://www.apache.org/dev/committers.html#committer-set-term Arvind Ralph On May 23, 2012, at 11:48 PM, Eric Sammer wrote: I appreciate your position Ralph and I don't want anyone to feel like they can't contribute. As we've talked about before, we've been quick to nurture new contributors to committer status successfully in a few cases. It's true that some of the more active committers are from Cloudera, but it's not to the exclusion of anyone. Others aren't from Cloudera. Those of us that work together are also very strict about abiding to the if it's not on the mailing list, it didn't happen rule (where mailing list can mean JIRA or other ASF infrastructure as well). I'm happy to take your guidance as a mentor, but you also need to understand that some of the ways the Flume project has elected to operate are just a matter of taste. They were proposed, discussed, voted on (and not as a block by Cloudera employees, IIRC - pretty sure I was -0), and put in place and do not violate the Apache Way (like RTC vs. CTR). They aren't unheard of and they do not work to the exclusion of contributors (RTC, for instance, only impacts committers). I think the vote that was started was only to gauge community opinion as a first step (although I'm not completely well versed in the graduation process, to be honest). If there are concrete things we can do to improve diversity, in your opinion, I am extremely open to hearing them. We already do many of the (excellent) things listed earlier in the thread. JIRA noise withstanding (again, it's a matter of taste - I use the email frequently as I find trolling through JIRA slow) I'm definitely open to ideas. Of course, if Flume simply needs to remain in the incubator until we develop greater diversity, that's fine too. If we're not ready, we're just not ready. On Wed, May 23, 2012 at 11:18 PM, Ralph Goers ralph.go...@dslextreme.comwrote: On May 23, 2012, at 10:48 PM, Patrick Hunt wrote: On Wed, May 23, 2012 at 10:36 PM, Ralph Goers ralph.go...@dslextreme.com wrote: On May 23, 2012, at 10:15 PM, Benson Margulies wrote: On Wed, May 23, 2012 at 10:09 PM, Ralph Goers ralph.go...@dslextreme.com wrote: Right after I read Jukka's email that started this thread and I posted my reply and discovered to my shock that they had started a graduation vote. I am shocked because I have pointed out repeatedly the project's complete lack of diversity. Virtually all the active PMC members and committers work for the same employer. I have told them several times that I would actually like to participate in the project but the way the project works is very different then every other project I am involved with at the ASF and the barriers
Re: Flume Graduation (was Re: June reports in two weeks)
On May 24, 2012, at 10:40 AM, Arvind Prabhakar wrote: Hi, On Thu, May 24, 2012 at 12:19 AM, Ralph Goers ralph.go...@dslextreme.comwrote: The ONLY issue I see for Flume to graduate is diversity. No one will convince me that the current makeup constitutes diversity of any kind. Perhaps I shouldn't have brought up the mailing list issues as that was only meant in the spirit of trying to offer some advice on how more diversity could be achieved. Flume is really the only community I participate in that contains Cloudera employees so I do find myself wondering if the way the project is run is because that is the way all projects with a large number of Cloudera employees are run. That might make all of those participants comfortable but might create a barrier to others. Here are the committers who have been active in the past three months: * Brock Noland (Cloudera) * Hari Shreedharan (Cloudera) * Jarek Jarcec Cecho (AVG Technologies) * Juhani Connolly (CyberAgent) * Mike Percy (Cloudera) * Mingjie Lai (Trend Micro) * Prasad Mujumdar (Cloudera) * Will McQueen (Cloudera) * Arvind Prabhakar (Cloudera) There are four companies represented in this list: AVG Technologies, Cloudera, CyberAgent and Trend Micro. Compared to other projects that have successfully graduated from Incubator in the past, this meets the diversity requirements very well. I was mistaken and the list above is indeed correct. For some reason I thought a couple of them had become Cloudera employees. However, none of those three are currently on the PPMC. When you look at the PPMC list you should also include a few more Cloudera people who do participate in release votes and PPMC issues. Most, if not all, of the non-Cloudera PMC members don't. In any case - I'm not insisting that the way the project is run needs to change. I'm simply saying I cannot support graduation with the current makeup of the committers and PMC. I don't have a hard and fast ratio - gaining 10 new unaffiliated committers who don't do much isn't nearly as good as 2 or 3 who are very active. Ultimately the project needs to figure out how to solve this. Stating that some committers who don't do much isn't nearly as good as 2 or 3 who are very active is an unfair characterization. This is more unfair for those who are part of the project but have not been active lately due to whatever reasons, but have played a foundational role in getting the project to a point where it is today. I think they are as important as any other committer who may be very active at the moment. Merit once earned, never expires [1]. [1] http://www.apache.org/dev/committers.html#committer-set-term I think you misunderstood my point or I didn't state it very well. Diversity isn't achieved simply by having bodies. IOW I am not suggesting offering commit rights to people who haven't earned it just to meet some ratio. However, I am not suggesting the project has ever even considered doing that. Ralph - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
On May 24, 2012, at 11:49 AM, Ralph Goers wrote: On May 24, 2012, at 10:40 AM, Arvind Prabhakar wrote: Hi, On Thu, May 24, 2012 at 12:19 AM, Ralph Goers ralph.go...@dslextreme.comwrote: The ONLY issue I see for Flume to graduate is diversity. No one will convince me that the current makeup constitutes diversity of any kind. Perhaps I shouldn't have brought up the mailing list issues as that was only meant in the spirit of trying to offer some advice on how more diversity could be achieved. Flume is really the only community I participate in that contains Cloudera employees so I do find myself wondering if the way the project is run is because that is the way all projects with a large number of Cloudera employees are run. That might make all of those participants comfortable but might create a barrier to others. Here are the committers who have been active in the past three months: * Brock Noland (Cloudera) * Hari Shreedharan (Cloudera) * Jarek Jarcec Cecho (AVG Technologies) * Juhani Connolly (CyberAgent) * Mike Percy (Cloudera) * Mingjie Lai (Trend Micro) * Prasad Mujumdar (Cloudera) * Will McQueen (Cloudera) * Arvind Prabhakar (Cloudera) There are four companies represented in this list: AVG Technologies, Cloudera, CyberAgent and Trend Micro. Compared to other projects that have successfully graduated from Incubator in the past, this meets the diversity requirements very well. I was mistaken and the list above is indeed correct. For some reason I thought a couple of them had become Cloudera employees. However, none of those three are currently on the PPMC. When you look at the PPMC list you should also include a few more Cloudera people who do participate in release votes and PPMC issues. Most, if not all, of the non-Cloudera PMC members don't. I started reading some of the Flume website and I think that when you go to the main Wiki page: https://cwiki.apache.org/confluence/display/FLUME/Index When you click on the Flume Cookbook the resource is at cloudera.org. http://archive.cloudera.com/cdh/3/flume/Cookbook/ This page lists flume-...@cloudera.org and is a file with a revision dated May 7, 2012. You can make you own conclusions, but it looks like podling resources need to be migrated to the ASF. Regards, Dave In any case - I'm not insisting that the way the project is run needs to change. I'm simply saying I cannot support graduation with the current makeup of the committers and PMC. I don't have a hard and fast ratio - gaining 10 new unaffiliated committers who don't do much isn't nearly as good as 2 or 3 who are very active. Ultimately the project needs to figure out how to solve this. Stating that some committers who don't do much isn't nearly as good as 2 or 3 who are very active is an unfair characterization. This is more unfair for those who are part of the project but have not been active lately due to whatever reasons, but have played a foundational role in getting the project to a point where it is today. I think they are as important as any other committer who may be very active at the moment. Merit once earned, never expires [1]. [1] http://www.apache.org/dev/committers.html#committer-set-term I think you misunderstood my point or I didn't state it very well. Diversity isn't achieved simply by having bodies. IOW I am not suggesting offering commit rights to people who haven't earned it just to meet some ratio. However, I am not suggesting the project has ever even considered doing that. Ralph - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
According to Clutch [1] the project has added 8 committers since it entered incubation. Regarding diversity, committers from over four organizations are actively involved in Flume development, which is pretty healthy. There does seem to be a need to have more diversity at the PPMC level, however, so that's something that could be worked on. Tom [1] http://incubator.apache.org/clutch.html On Thu, May 24, 2012 at 2:06 PM, Dave Fisher dave2w...@comcast.net wrote: On May 24, 2012, at 11:49 AM, Ralph Goers wrote: On May 24, 2012, at 10:40 AM, Arvind Prabhakar wrote: Hi, On Thu, May 24, 2012 at 12:19 AM, Ralph Goers ralph.go...@dslextreme.comwrote: The ONLY issue I see for Flume to graduate is diversity. No one will convince me that the current makeup constitutes diversity of any kind. Perhaps I shouldn't have brought up the mailing list issues as that was only meant in the spirit of trying to offer some advice on how more diversity could be achieved. Flume is really the only community I participate in that contains Cloudera employees so I do find myself wondering if the way the project is run is because that is the way all projects with a large number of Cloudera employees are run. That might make all of those participants comfortable but might create a barrier to others. Here are the committers who have been active in the past three months: * Brock Noland (Cloudera) * Hari Shreedharan (Cloudera) * Jarek Jarcec Cecho (AVG Technologies) * Juhani Connolly (CyberAgent) * Mike Percy (Cloudera) * Mingjie Lai (Trend Micro) * Prasad Mujumdar (Cloudera) * Will McQueen (Cloudera) * Arvind Prabhakar (Cloudera) There are four companies represented in this list: AVG Technologies, Cloudera, CyberAgent and Trend Micro. Compared to other projects that have successfully graduated from Incubator in the past, this meets the diversity requirements very well. I was mistaken and the list above is indeed correct. For some reason I thought a couple of them had become Cloudera employees. However, none of those three are currently on the PPMC. When you look at the PPMC list you should also include a few more Cloudera people who do participate in release votes and PPMC issues. Most, if not all, of the non-Cloudera PMC members don't. I started reading some of the Flume website and I think that when you go to the main Wiki page: https://cwiki.apache.org/confluence/display/FLUME/Index When you click on the Flume Cookbook the resource is at cloudera.org. http://archive.cloudera.com/cdh/3/flume/Cookbook/ This page lists flume-...@cloudera.org and is a file with a revision dated May 7, 2012. You can make you own conclusions, but it looks like podling resources need to be migrated to the ASF. Regards, Dave In any case - I'm not insisting that the way the project is run needs to change. I'm simply saying I cannot support graduation with the current makeup of the committers and PMC. I don't have a hard and fast ratio - gaining 10 new unaffiliated committers who don't do much isn't nearly as good as 2 or 3 who are very active. Ultimately the project needs to figure out how to solve this. Stating that some committers who don't do much isn't nearly as good as 2 or 3 who are very active is an unfair characterization. This is more unfair for those who are part of the project but have not been active lately due to whatever reasons, but have played a foundational role in getting the project to a point where it is today. I think they are as important as any other committer who may be very active at the moment. Merit once earned, never expires [1]. [1] http://www.apache.org/dev/committers.html#committer-set-term I think you misunderstood my point or I didn't state it very well. Diversity isn't achieved simply by having bodies. IOW I am not suggesting offering commit rights to people who haven't earned it just to meet some ratio. However, I am not suggesting the project has ever even considered doing that. Ralph - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Flume Graduation (was Re: June reports in two weeks)
Right after I read Jukka's email that started this thread and I posted my reply and discovered to my shock that they had started a graduation vote. I am shocked because I have pointed out repeatedly the project's complete lack of diversity. Virtually all the active PMC members and committers work for the same employer. I have told them several times that I would actually like to participate in the project but the way the project works is very different then every other project I am involved with at the ASF and the barriers to figure out what is actually going on is very high. Almost nothing is discussed directly on the dev list - it is all done through Jira issues or the Review tool. While all the Jira issue updates and reviews are sent to the dev list most of that is just noise. Feel free to review the dev list archives to see what I am talking about. Needless to say, when the graduation proposal reaches this list, and I'm sure it will, I will strongly endorse the IPMC to reject the proposal. FWIW, I found the post below to be 100% on target. Ralph On May 23, 2012, at 7:31 PM, Marvin Humphrey wrote: On Wed, May 23, 2012 at 5:36 PM, Patrick Hunt ph...@apache.org wrote: Perhaps someone will have some insight on how to gather new contributors that hasn't been tried yet? Jukka's written on this subject multiple times in the past. Here are two gems, one from a while back, the other recent: http://markmail.org/message/o3gbgam4ny2upqte Most of the cases I've been involved so far of podlings in the hoping some more people come along have had symptoms of the project team not paying enough attention on making it easy for new contributors to show up and stick around. Things like complex and undocumented build steps, missing Getting started or Getting involved guides, lack of quick and positive feedback to newcomers, etc., are all too common. Fixing even just some of such things will dramatically increase the odds of new people showing up. Those are things that are very easy to overlook when you're working on your first open source projects (it took me years to learn those lessons), but we here have a massive amount of collective experience on such things. That's what we could and IMHO should be sharing with the podlings. That's what mentoring to me is about and that's where our most precious added value is. Otherwise incubation just boils down to an indoctrination period on how to apply and conform to the various Apache rules and policies. http://incubator.markmail.org/thread/qpzg6wdoq7cwko55 I've been involved with quite a few podlings with similar problems in attracting longer-term contributors. In my experience the best way to solve that problem is to change your mindset of expecting most such people to be just one-off contributors. If you instead treat them as your next new committers and engage with them as peers, many (though of course not all) will respond in kind and actually become more involved. Many developers, especially from commercial backgrounds, tend to treat such contributors as just users reporting a problem. A typical interaction goes like What's the problem? Do you have a test case? OK, let me fix it (when I get around to it). A better approach is something like What's the problem? OK, here are some pointers to the relevant bits in code. How do you think this should be fixed? Here's another tip I picked up from Joe Schaefer: when you're voting in a new committer and they have a big patch set sitting in the queue, hold off and let *them* commit it so that they get the satisfaction, the new experience, and the appreciation all at once. It would be nice if stuff like this was collected in Steps to building a community documentation somewhere, rather than scattered through the email archives. I suggest Steps as a format because different approaches are required at different phases of the project and sizes of the community. Marvin Humphrey - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Flume Graduation (was Re: June reports in two weeks)
On Wed, May 23, 2012 at 10:09 PM, Ralph Goers ralph.go...@dslextreme.com wrote: Right after I read Jukka's email that started this thread and I posted my reply and discovered to my shock that they had started a graduation vote. I am shocked because I have pointed out repeatedly the project's complete lack of diversity. Virtually all the active PMC members and committers work for the same employer. I have told them several times that I would actually like to participate in the project but the way the project works is very different then every other project I am involved with at the ASF and the barriers to figure out what is actually going on is very high. Almost nothing is discussed directly on the dev list - it is all done through Jira issues or the Review tool. While all the Jira issue updates and reviews are sent to the dev list most of that is just noise. Feel free to review the dev list archives to see what I am talking about. I don't follow flume, but I'd propose to soften your objection only slightly. I've met other groups of people who like a JIRA centric view of the world. I suspect that if they did a bunch of other good things called out below, you or others would find the JIRA business digestible. Also, on the other hand, I fear that the co-employed contributors are collaborating in the hallway, and the lack of the context in JIRA or on the list is contributing to the problem. Needless to say, when the graduation proposal reaches this list, and I'm sure it will, I will strongly endorse the IPMC to reject the proposal. FWIW, I found the post below to be 100% on target. Ralph On May 23, 2012, at 7:31 PM, Marvin Humphrey wrote: On Wed, May 23, 2012 at 5:36 PM, Patrick Hunt ph...@apache.org wrote: Perhaps someone will have some insight on how to gather new contributors that hasn't been tried yet? Jukka's written on this subject multiple times in the past. Here are two gems, one from a while back, the other recent: http://markmail.org/message/o3gbgam4ny2upqte Most of the cases I've been involved so far of podlings in the hoping some more people come along have had symptoms of the project team not paying enough attention on making it easy for new contributors to show up and stick around. Things like complex and undocumented build steps, missing Getting started or Getting involved guides, lack of quick and positive feedback to newcomers, etc., are all too common. Fixing even just some of such things will dramatically increase the odds of new people showing up. Those are things that are very easy to overlook when you're working on your first open source projects (it took me years to learn those lessons), but we here have a massive amount of collective experience on such things. That's what we could and IMHO should be sharing with the podlings. That's what mentoring to me is about and that's where our most precious added value is. Otherwise incubation just boils down to an indoctrination period on how to apply and conform to the various Apache rules and policies. http://incubator.markmail.org/thread/qpzg6wdoq7cwko55 I've been involved with quite a few podlings with similar problems in attracting longer-term contributors. In my experience the best way to solve that problem is to change your mindset of expecting most such people to be just one-off contributors. If you instead treat them as your next new committers and engage with them as peers, many (though of course not all) will respond in kind and actually become more involved. Many developers, especially from commercial backgrounds, tend to treat such contributors as just users reporting a problem. A typical interaction goes like What's the problem? Do you have a test case? OK, let me fix it (when I get around to it). A better approach is something like What's the problem? OK, here are some pointers to the relevant bits in code. How do you think this should be fixed? Here's another tip I picked up from Joe Schaefer: when you're voting in a new committer and they have a big patch set sitting in the queue, hold off and let *them* commit it so that they get the satisfaction, the new experience, and the appreciation all at once. It would be nice if stuff like this was collected in Steps to building a community documentation somewhere, rather than scattered through the email archives. I suggest Steps as a format because different approaches are required at different phases of the project and sizes of the community. Marvin Humphrey - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail:
Re: Flume Graduation (was Re: June reports in two weeks)
On May 23, 2012, at 10:15 PM, Benson Margulies wrote: On Wed, May 23, 2012 at 10:09 PM, Ralph Goers ralph.go...@dslextreme.com wrote: Right after I read Jukka's email that started this thread and I posted my reply and discovered to my shock that they had started a graduation vote. I am shocked because I have pointed out repeatedly the project's complete lack of diversity. Virtually all the active PMC members and committers work for the same employer. I have told them several times that I would actually like to participate in the project but the way the project works is very different then every other project I am involved with at the ASF and the barriers to figure out what is actually going on is very high. Almost nothing is discussed directly on the dev list - it is all done through Jira issues or the Review tool. While all the Jira issue updates and reviews are sent to the dev list most of that is just noise. Feel free to review the dev list archives to see what I am talking about. I don't follow flume, but I'd propose to soften your objection only slightly. I've met other groups of people who like a JIRA centric view of the world. I suspect that if they did a bunch of other good things called out below, you or others would find the JIRA business digestible. Also, on the other hand, I fear that the co-employed contributors are collaborating in the hallway, and the lack of the context in JIRA or on the list is contributing to the problem. I have reason to doubt the collaboration in the hallway aspect and I certainly do not doubt everyone's good intent. I'm not objecting to the collaboration style as an issue preventing graduation. I'm just saying I find it difficult to participate with that style and that simply makes me wonder if that is making it harder to attract new committers. I fully realize that that issue might just be with me, but the fact remains that there is practically no diversity in the project and I cannot in good conscience recommend graduation for a project in that situation. Ralph Needless to say, when the graduation proposal reaches this list, and I'm sure it will, I will strongly endorse the IPMC to reject the proposal. FWIW, I found the post below to be 100% on target. Ralph On May 23, 2012, at 7:31 PM, Marvin Humphrey wrote: On Wed, May 23, 2012 at 5:36 PM, Patrick Hunt ph...@apache.org wrote: Perhaps someone will have some insight on how to gather new contributors that hasn't been tried yet? Jukka's written on this subject multiple times in the past. Here are two gems, one from a while back, the other recent: http://markmail.org/message/o3gbgam4ny2upqte Most of the cases I've been involved so far of podlings in the hoping some more people come along have had symptoms of the project team not paying enough attention on making it easy for new contributors to show up and stick around. Things like complex and undocumented build steps, missing Getting started or Getting involved guides, lack of quick and positive feedback to newcomers, etc., are all too common. Fixing even just some of such things will dramatically increase the odds of new people showing up. Those are things that are very easy to overlook when you're working on your first open source projects (it took me years to learn those lessons), but we here have a massive amount of collective experience on such things. That's what we could and IMHO should be sharing with the podlings. That's what mentoring to me is about and that's where our most precious added value is. Otherwise incubation just boils down to an indoctrination period on how to apply and conform to the various Apache rules and policies. http://incubator.markmail.org/thread/qpzg6wdoq7cwko55 I've been involved with quite a few podlings with similar problems in attracting longer-term contributors. In my experience the best way to solve that problem is to change your mindset of expecting most such people to be just one-off contributors. If you instead treat them as your next new committers and engage with them as peers, many (though of course not all) will respond in kind and actually become more involved. Many developers, especially from commercial backgrounds, tend to treat such contributors as just users reporting a problem. A typical interaction goes like What's the problem? Do you have a test case? OK, let me fix it (when I get around to it). A better approach is something like What's the problem? OK, here are some pointers to the relevant bits in code. How do you think this should be fixed? Here's another tip I picked up from Joe Schaefer: when you're voting in a new committer and they have a big patch set sitting in the queue, hold off and let *them* commit it so that they get the satisfaction, the new experience, and the
Re: Flume Graduation (was Re: June reports in two weeks)
On Wed, May 23, 2012 at 10:36 PM, Ralph Goers ralph.go...@dslextreme.com wrote: On May 23, 2012, at 10:15 PM, Benson Margulies wrote: On Wed, May 23, 2012 at 10:09 PM, Ralph Goers ralph.go...@dslextreme.com wrote: Right after I read Jukka's email that started this thread and I posted my reply and discovered to my shock that they had started a graduation vote. I am shocked because I have pointed out repeatedly the project's complete lack of diversity. Virtually all the active PMC members and committers work for the same employer. I have told them several times that I would actually like to participate in the project but the way the project works is very different then every other project I am involved with at the ASF and the barriers to figure out what is actually going on is very high. Almost nothing is discussed directly on the dev list - it is all done through Jira issues or the Review tool. While all the Jira issue updates and reviews are sent to the dev list most of that is just noise. Feel free to review the dev list archives to see what I am talking about. I don't follow flume, but I'd propose to soften your objection only slightly. I've met other groups of people who like a JIRA centric view of the world. I suspect that if they did a bunch of other good things called out below, you or others would find the JIRA business digestible. Also, on the other hand, I fear that the co-employed contributors are collaborating in the hallway, and the lack of the context in JIRA or on the list is contributing to the problem. I have reason to doubt the collaboration in the hallway aspect and I certainly do not doubt everyone's good intent. I'm not objecting to the collaboration style as an issue preventing graduation. I'm just saying I find it difficult to participate with that style and that simply makes me wonder if that is making it harder to attract new committers. I fully realize that that issue might just be with me, but the fact remains that there is practically no diversity in the project and I cannot in good conscience recommend graduation for a project in that situation. Hi Ralph, Benson, et. al., some background: Flume is similar to Hadoop and other related projects in that it is very jira heavy for development activity. No slouch in terms of mailing list traffic either though (1200 last month): http://flume.markmail.org/ Also note the extensive new developer type detail that's available on the web/wiki: https://cwiki.apache.org/confluence/display/FLUME/Index The team list can provide insight into the diversity issue http://incubator.apache.org/flume/team-list.html My understanding is that there are at least 4 separate organizations represented by active commiters. Regards, Patrick Needless to say, when the graduation proposal reaches this list, and I'm sure it will, I will strongly endorse the IPMC to reject the proposal. FWIW, I found the post below to be 100% on target. Ralph On May 23, 2012, at 7:31 PM, Marvin Humphrey wrote: On Wed, May 23, 2012 at 5:36 PM, Patrick Hunt ph...@apache.org wrote: Perhaps someone will have some insight on how to gather new contributors that hasn't been tried yet? Jukka's written on this subject multiple times in the past. Here are two gems, one from a while back, the other recent: http://markmail.org/message/o3gbgam4ny2upqte Most of the cases I've been involved so far of podlings in the hoping some more people come along have had symptoms of the project team not paying enough attention on making it easy for new contributors to show up and stick around. Things like complex and undocumented build steps, missing Getting started or Getting involved guides, lack of quick and positive feedback to newcomers, etc., are all too common. Fixing even just some of such things will dramatically increase the odds of new people showing up. Those are things that are very easy to overlook when you're working on your first open source projects (it took me years to learn those lessons), but we here have a massive amount of collective experience on such things. That's what we could and IMHO should be sharing with the podlings. That's what mentoring to me is about and that's where our most precious added value is. Otherwise incubation just boils down to an indoctrination period on how to apply and conform to the various Apache rules and policies. http://incubator.markmail.org/thread/qpzg6wdoq7cwko55 I've been involved with quite a few podlings with similar problems in attracting longer-term contributors. In my experience the best way to solve that problem is to change your mindset of expecting most such people to be just one-off contributors. If you instead treat them as your next new committers and engage with them as peers, many (though of course not all) will respond in kind and
