fwiw - i agree with Dhruba's take. i can't imagine (for example) hadoop shipping without hadoop-streaming (which is contrib). and it's awesome that hadoop changes that break streaming are caught early and often.
it makes sense this analog doesn't apply to all contrib projects - perhaps they shouldn't be in contrib to begin with? alternatively - to extend Dhruba's idea - one could break up contrib projects into those that are critical (like streaming would be to hadoop and i imagine the replication stuff would be to hbase) - and those that are non-critical (and not build these by default). the only reason to have non-critical contrib projects in the codebase seems to be to give young/new projects exposure. i am not sure whether this is a good enough reason - perhaps those with some historical perspective have better idea on this. On Thu, Jan 28, 2010 at 2:35 PM, Ryan Rawson <ryano...@gmail.com> wrote: > Part of the problem with contrib is fixing the deeper problems during > major version changes (eg: new file format in 0.20) requires > specialized knowledge that only the contrib author has. Then they must > write a patch, submit it to a hbase committer, etc. If the contrib > was in a non-ASF repo, the author could fix it much faster. > > That's my theory at least? > > On Thu, Jan 28, 2010 at 2:27 PM, Dhruba Borthakur <dhr...@gmail.com> wrote: >> Hi stack, >> >> I have seen this issue crop up in the HDFS repository too. >> >> The issue that you have brought up is that code changes to core sometimes >> needs lots of changes to contrib to make it compile. This is a big problem, >> especially when all developers are not intricately familiar with each and >> every contrib module. However, the big plus is that when a HBase release is >> made, it has all the contrib modules in it, and it makes it easier for a >> user to start using any of those contrib modules right away. Can we get the >> best of both worlds if we do something like this: >> >> Contribs are not first-class citizens. The build script can be setup to >> ignore build failures in contrib modules. The hope is that when a release is >> about to be cut, contrib owners will submit fixes to any build problems if >> they want their contrib to be shipped as part of the Hbase release. will >> this solve your problem? >> >> thanks, >> dhruba >> >> >> >> On Thu, Jan 28, 2010 at 12:42 PM, Ryan Rawson <ryano...@gmail.com> wrote: >> >>> +1 >>> >>> I think when we are ready, we should publish the replication as a >>> contrib up on github. I really like github, its way better than any >>> other system I've seen (in part due to git of course). >>> >>> I do like the notion of officially released contribs that are included >>> at specific version numbers with the hbase release. Hbase core 0.21.0 >>> with TH, ITH at version 0.1 or whatever. >>> >>> -ryan >>> >>> On Thu, Jan 28, 2010 at 12:38 PM, Stack <st...@duboce.net> wrote: >>> > I'd like to start a discussion on moving src/contrib out of hbase. >>> > Keep reading if you have an opinion. >>> > >>> > I'd like to suggest that we undo the notion of hbase contribs for the >>> > following reasons: >>> > >>> > + In my experience, they present a friction on changes to core as any >>> > significant core change tends to ripple down into the contribs whether >>> > its code or infrastructure/build changes. Usually what happens then >>> > is a non-expert in the contrib code is making edits -- often radical >>> > -- to code they are not completely up on and are a little frustrated >>> > that they have to do it. Bad. >>> > + A few of our contribs are maintained by non-committers. This means >>> > it takes a committers time getting in updates. The owner is at mercy >>> > of the committer making wanted changes. The committer is consumed >>> > reviewing and making update. This indirection hurts at both ends (We >>> > could discuss making contrib owners committers on their contrib only >>> > but that'd be a bit of bureaucratic nightmare and a burden on the >>> > hadoop pmc to vote on granting access to a subprojects, contrib. Its >>> > tough enough getting hadoop pmc to vote on hbase committers. There is >>> > no precedent in other project, to my knowledge). >>> > + Contribs and core evolve at different rates. They should not be >>> > constrained by core release schedule (or the opposite, core should not >>> > be held up because fixes in contrib are wanting). >>> > >>> > I suggest that current contribs be moved out of hbase up to standalone >>> > github or google code projects (witness how hbql does it). Previous >>> > to our move to Ivy (and possibly soon, Maven), asking contribs be >>> > standalone was a pain as they'd have to check in hbase jars and all of >>> > dependencies and then move these forward over time. Now that we are >>> > Ivy-ized, contribs just need write a bit of ivy.xml and it'll take >>> > care of pulling dependencies. >>> > >>> > I'd imagine support for contribs would go on as it does now with >>> > queries up on hbase mailing list and help out on IRC. We'd give >>> > contribs first-class billing up on home page. Popular contribs might >>> > run their own mailing lists, etc... >>> > >>> > We should chat but some contribs should be pulled up into core. >>> > Thrift was core. Talk was to move current thrift update out to >>> > contrib. I think now it should just stay in core. Stargate perhaps >>> > should come up into core? >>> > >>> > What do folks think? >>> > St.Ack >>> > >>> > P.S. I've fostered the contrib notion in the past. I've since had a >>> > change of heart. Please pardon my flip. >>> > >>> >> >> >> >> -- >> Connect to me at http://www.facebook.com/dhruba >> >
