What does it take to move a contrib into core? I would think that a committer must be able to fix any problem with it at minimum...
On Sat, Jan 30, 2010 at 4:57 PM, Stack <st...@duboce.net> wrote: > On Thu, Jan 28, 2010 at 3:08 PM, Joydeep Sarma <jsensa...@gmail.com> wrote: >> fwiw - i agree with Dhruba's take. i can't imagine (for example) >> hadoop shipping without hadoop-streaming (which is contrib). and it's >> awesome that hadoop changes that break streaming are caught early and >> often. > > I suggest that contribs we deem critical should be pulled up out of > contrib and moved up to core. > > Regards changes in core breaking contrib early, contribs moved out to > other repositories can retain this 'benefit' by building against > snapshots of the core TRUNK via built-in ivy/maven facility, only now, > the onus for updating the moved contrib is no longer inline with core > dev and instead has been shifted to the moved contrib maintainer. It > is up to him or her to update to match core changes at their pleasure > or to report on core changes that cause fundamental distress in > contrib. > > >> it makes sense this analog doesn't apply to all contrib projects - >> perhaps they shouldn't be in contrib to begin with? > > Yes. There is some of this going on. IMO, all currently have > sufficient character living apart outside contrib. > > > alternatively - to >> extend Dhruba's idea - one could break up contrib projects into those >> that are critical (like streaming would be to hadoop and i imagine the >> replication stuff would be to hbase) - and those that are non-critical >> (and not build these by default). >> > We could do this but to my mind, I can't help but think that we are > just piling the work integrating contribs to the period that just > precedes a release. > > >> the only reason to have non-critical contrib projects in the codebase >> seems to be to give young/new projects exposure. i am not sure whether >> this is a good enough reason - perhaps those with some historical >> perspective have better idea on this. >> > > You have it right. I think it fair to say that that explains how we > ended up with our current set of contribs. At least one contrib has > been in hbase more than a year now. Its time we held a graduation for > it at least? > > St.Ack > >> >> On Thu, Jan 28, 2010 at 2:35 PM, Ryan Rawson <ryano...@gmail.com> wrote: >>> Part of the problem with contrib is fixing the deeper problems during >>> major version changes (eg: new file format in 0.20) requires >>> specialized knowledge that only the contrib author has. Then they must >>> write a patch, submit it to a hbase committer, etc. If the contrib >>> was in a non-ASF repo, the author could fix it much faster. >>> >>> That's my theory at least? >>> >>> On Thu, Jan 28, 2010 at 2:27 PM, Dhruba Borthakur <dhr...@gmail.com> wrote: >>>> Hi stack, >>>> >>>> I have seen this issue crop up in the HDFS repository too. >>>> >>>> The issue that you have brought up is that code changes to core sometimes >>>> needs lots of changes to contrib to make it compile. This is a big problem, >>>> especially when all developers are not intricately familiar with each and >>>> every contrib module. However, the big plus is that when a HBase release is >>>> made, it has all the contrib modules in it, and it makes it easier for a >>>> user to start using any of those contrib modules right away. Can we get the >>>> best of both worlds if we do something like this: >>>> >>>> Contribs are not first-class citizens. The build script can be setup to >>>> ignore build failures in contrib modules. The hope is that when a release >>>> is >>>> about to be cut, contrib owners will submit fixes to any build problems if >>>> they want their contrib to be shipped as part of the Hbase release. will >>>> this solve your problem? >>>> >>>> thanks, >>>> dhruba >>>> >>>> >>>> >>>> On Thu, Jan 28, 2010 at 12:42 PM, Ryan Rawson <ryano...@gmail.com> wrote: >>>> >>>>> +1 >>>>> >>>>> I think when we are ready, we should publish the replication as a >>>>> contrib up on github. I really like github, its way better than any >>>>> other system I've seen (in part due to git of course). >>>>> >>>>> I do like the notion of officially released contribs that are included >>>>> at specific version numbers with the hbase release. Hbase core 0.21.0 >>>>> with TH, ITH at version 0.1 or whatever. >>>>> >>>>> -ryan >>>>> >>>>> On Thu, Jan 28, 2010 at 12:38 PM, Stack <st...@duboce.net> wrote: >>>>> > I'd like to start a discussion on moving src/contrib out of hbase. >>>>> > Keep reading if you have an opinion. >>>>> > >>>>> > I'd like to suggest that we undo the notion of hbase contribs for the >>>>> > following reasons: >>>>> > >>>>> > + In my experience, they present a friction on changes to core as any >>>>> > significant core change tends to ripple down into the contribs whether >>>>> > its code or infrastructure/build changes. Usually what happens then >>>>> > is a non-expert in the contrib code is making edits -- often radical >>>>> > -- to code they are not completely up on and are a little frustrated >>>>> > that they have to do it. Bad. >>>>> > + A few of our contribs are maintained by non-committers. This means >>>>> > it takes a committers time getting in updates. The owner is at mercy >>>>> > of the committer making wanted changes. The committer is consumed >>>>> > reviewing and making update. This indirection hurts at both ends (We >>>>> > could discuss making contrib owners committers on their contrib only >>>>> > but that'd be a bit of bureaucratic nightmare and a burden on the >>>>> > hadoop pmc to vote on granting access to a subprojects, contrib. Its >>>>> > tough enough getting hadoop pmc to vote on hbase committers. There is >>>>> > no precedent in other project, to my knowledge). >>>>> > + Contribs and core evolve at different rates. They should not be >>>>> > constrained by core release schedule (or the opposite, core should not >>>>> > be held up because fixes in contrib are wanting). >>>>> > >>>>> > I suggest that current contribs be moved out of hbase up to standalone >>>>> > github or google code projects (witness how hbql does it). Previous >>>>> > to our move to Ivy (and possibly soon, Maven), asking contribs be >>>>> > standalone was a pain as they'd have to check in hbase jars and all of >>>>> > dependencies and then move these forward over time. Now that we are >>>>> > Ivy-ized, contribs just need write a bit of ivy.xml and it'll take >>>>> > care of pulling dependencies. >>>>> > >>>>> > I'd imagine support for contribs would go on as it does now with >>>>> > queries up on hbase mailing list and help out on IRC. We'd give >>>>> > contribs first-class billing up on home page. Popular contribs might >>>>> > run their own mailing lists, etc... >>>>> > >>>>> > We should chat but some contribs should be pulled up into core. >>>>> > Thrift was core. Talk was to move current thrift update out to >>>>> > contrib. I think now it should just stay in core. Stargate perhaps >>>>> > should come up into core? >>>>> > >>>>> > What do folks think? >>>>> > St.Ack >>>>> > >>>>> > P.S. I've fostered the contrib notion in the past. I've since had a >>>>> > change of heart. Please pardon my flip. >>>>> > >>>>> >>>> >>>> >>>> >>>> -- >>>> Connect to me at http://www.facebook.com/dhruba >>>> >>> >> >
