On 25 May 2012 20:00, Josh Wills <jwi...@cloudera.com> wrote: > Hi Steve, > > Thank you for your thoughtful comments. Replies inlined below. > > > 1. He's using it at work, so represents the end users. > > A super-majority of the initial committers are also end users. I use > Crunch on my own projects (e.g., > http://github.com/cloudera/seismichadoop and > http://github.com/cloudera/matching ), Cloudera solutions architects > use Crunch on client projects, Robert is building tools on top of > Crunch at WibiData, and Gabriel and Chris use it for building > pipelines at TomTom. I can't speak for Tom and Vinod, but of course, > they have other positive qualities. :) >
It's still a fairly limited set of organisations and lacks independence. Jakob and colleagues have no strategic goals in ensuring the success/failure of any specific OSS project, merely getting the right tools for their job. In a true OSS project -not one that is released under some OSS license but is effectively a single-vendor-project (JBoss, MySQL, etc) - end users are not merely consumers of the output, they are potential engineering resources to be co-opted, be it in their suggestions for improvement, documentation, bugreps, tests and code itself. That's the challenge -and it's not easy, especially in a project where some of the developers work on it full time, others are people that use it a bit and find bugs. Those little contributors need to be nurtured until they become good ones. > > > 2. His code is always of high quality > > I in no way meant to disparage Jakob or his coding. The objective of > my reply was say "no" in the most apologetic, obsequious way possible > while not going so far over the top as to sound insincere. Having > LinkedIn on board would be a tremendous PR boost for the project. It > was painful to say no. > > I am in no way savvy in the ways of Apache or the politics of the ASF. > Not Apache politics, but a core belief: the notion that a community is actually more important than the codebase itself. The goal of an incubating project is not so much to get into the code into a shape where it is ready to graduate -but build a community to a point where it is considered successful. If you don't want that, but instead want to have a project over which you retain tight control, you are free to continue to host it on github. > I understand that smart people who I respect a great deal think that > this is the wrong decision. But I think that it takes something really > great for someone to see a project like Crunch, play around with, and > then take the time to make some contributions to it without any > expectation of recognition, in the form of an Apache committership or > anything else. That was what Gabriel and Chris and Robert did over the > past few months. I really admire that, and I think that it deserves > some special recognition, however small. That is good, and their past and hopefully ongoing work will help the project -I just think that it would have helped the project if Jakob's had been embraced > I'm willing to have some > people not like me or think I'm dumb if that's the price of giving > that to them. > I all I have are concerns that the proposal is at risk from the same problems that others have had in incubation. > > 3. Given the ongoing discussion on diversity w.r.t Flume, I think it > would > > be wise to not follow that projects example, and try to get broader > > involvement from the outset. > > I agree that it is critical to have broad involvement at the outset. > Both S4 and Flume started out with at least 50% of their initial > committers from a single company, and no single company constitutes a > majority of the initial committers to Crunch (Cloudera has three, > TomTom has two, WibiData has one, and Hortonworks has one). That de > jure diversity mirrors the de facto diversity in Crunch's commit logs > over the past several months: > > https://github.com/cloudera/crunch/commits/master > > There is nothing more important than increasing that de facto > diversity over time. I fully expect that my role during the incubator > process is to be the best documenter, repository maintainer, and > recruiter of new contributors that I can be. > It's not clear that Flume has widened its developer base significantly enough for it to graduate. I fear that Crunch is exposed to the same risks, and the fact that you are opting to exclude Jakob from the initial dev team concerns me. -Steve
