On Tue, Jul 24, 2018 at 10:01 PM Misty Linville <[email protected]> wrote:
> I like the idea of a separate connectors repo/release vehicle, but I'm a > little concerned about the need to release all together to update just one > of the connectors. How would that work? What kind of compatibility > guarantees are we signing up for? > > I hate responses that begin "Good question" -- so fawning -- but, ahem, good question Misty (in the literal, not flattering, sense). I think hbase-connectors will be like hbase-thirdparty. The latter includes netty, pb, guava and a few other bits and pieces so yeah, sometimes a netty upgrade or an improvement on our patch to pb will require us releasing all though we are fixing one lib only. Usually, if bothering to make a release, we'll check for fixes or updates we can do in the other bundled components. On the rate of releases, I foresee a flurry of activity around launch as we fill missing bits and address critical bug fixes, but that then it will settle down to be boring, with just the occasional update. Thrift and REST have been stable for a good while now (not saying this is a good thing). Our Sean just suggested moving mapreduce to connectors too -- an interesting idea -- and this has also been stable too (at least until recently with the shading work). We should talk about the Spark connector when it comes time. It might not be as stable as the others. On the compatibility guarantees, we'll semver it so if an incompatible change in a connector or if the connectors have to change to match a new version of hbase, we'll make sure the hbase-connector version number is changed appropriately. On the backend, what Mike says; connectors use HBase Public APIs (else they can't be moved to the hbase-connector repo). S > On Tue, Jul 24, 2018, 9:41 PM Stack <[email protected]> wrote: > > > Grand. I filed https://issues.apache.org/jira/browse/HBASE-20934. Let me > > have a go at making the easy one work first (the kafka proxy). Lets see > how > > it goes. I'll report back here. > > S > > > > On Tue, Jul 24, 2018 at 2:43 PM Sean Busbey <[email protected]> wrote: > > > > > Key functionality for the project's adoption should be in the project. > > > Please do not suggest we donate things to Bahir. > > > > > > I apologize if this is brisk. I have had previous negative experiences > > > with folks that span our communities trying to move work I spent a lot > > > of time contributing to within HBase over to Bahir in an attempt to > > > bypass an agreed upon standard of quality. > > > > > > On Tue, Jul 24, 2018 at 3:38 PM, Artem Ervits <[email protected]> > > > wrote: > > > > Why not just donating the connector to http://bahir.apache.org/ ? > > > > > > > > On Tue, Jul 24, 2018, 12:51 PM Lars Francke <[email protected]> > > > wrote: > > > > > > > >> I'd love to have the Kafka Connector included. > > > >> > > > >> @Mike thanks so much for the contribution (and your planned ones) > > > >> > > > >> I'm +1 on adding it to the core but I'm also +1 on having a separate > > > >> repository under Apache governance > > > >> > > > >> On Tue, Jul 24, 2018 at 6:01 PM, Josh Elser <[email protected]> > > wrote: > > > >> > > > >> > +1 to the great point by Duo about use of non-IA.Public classes > > > >> > > > > >> > +1 for Apache for the governance (although, I wouldn't care if we > > use > > > >> > Github PRs to try to encourage more folks to contribute), a repo > > with > > > the > > > >> > theme of "connectors" (to include Thrift, REST, and the like). > Spark > > > too > > > >> -- > > > >> > I think we had suggested that prior, but it could be a mental > > > invention > > > >> of > > > >> > mine.. > > > >> > > > > >> > > > > >> > On 7/24/18 10:16 AM, Hbase Janitor wrote: > > > >> > > > > >> >> Hi everyone, > > > >> >> > > > >> >> I'm the author of the patch. A separate repo for all the > > connectors > > > is > > > >> a > > > >> >> great idea! I can make whatever changes necessary to the patch to > > > help. > > > >> >> > > > >> >> I have several other integration type projects like this planned. > > > >> >> > > > >> >> Mike > > > >> >> > > > >> >> > > > >> >> On Tue, Jul 24, 2018, 00:03 Mike Drob <[email protected]> wrote: > > > >> >> > > > >> >> I would be ok with all of the connectors in a single repo. Doing > a > > > repo > > > >> >>> per > > > >> >>> connector seems like a large amount of overhead work. > > > >> >>> > > > >> >>> On Mon, Jul 23, 2018, 9:12 PM Clay B. <[email protected]> wrote: > > > >> >>> > > > >> >>> [Non-binding] > > > >> >>>> > > > >> >>>> I am all for the Kafka Connect(er) as indeed it makes HBase > "more > > > >> >>>> relevant" and generates buzz to help me sell HBase adoption in > my > > > >> >>>> endeavors. > > > >> >>>> > > > >> >>>> Also, I would like to see a connectors repo a lot as I would > > > expect it > > > >> >>>> > > > >> >>> can > > > >> >>> > > > >> >>>> make the HBase source and releases more obvious in what is > > > changing. > > > >> Not > > > >> >>>> to distract from Kafka, but Spark has in the past been a > hang-up > > > and > > > >> >>>> > > > >> >>> seems > > > >> >>> > > > >> >>>> a good fit in such a repo too; as such, I would prefer Apache > > over > > > >> >>>> > > > >> >>> GitHub. > > > >> >>> > > > >> >>>> > > > >> >>>> -Clay > > > >> >>>> > > > >> >>>> On Mon, 23 Jul 2018, Andrew Purtell wrote: > > > >> >>>> > > > >> >>>> Would we make a new repo called hbase-connectors and move REST, > > > >> >>>>>> > > > >> >>>>> thrift, > > > >> >>> > > > >> >>>> and this new patch there? > > > >> >>>>> > > > >> >>>>> I like this idea. We are already releasing hbase-thirdparty > like > > > >> this. > > > >> >>>>> > > > >> >>>>> > > > >> >>>>> On Mon, Jul 23, 2018 at 5:47 PM Stack <[email protected]> > wrote: > > > >> >>>>> > > > >> >>>>> (Thanks for the good discussion) > > > >> >>>>>> > > > >> >>>>>> Where we think 'outside of HBase' would be? > > > >> >>>>>> > > > >> >>>>>> Github seems too 'remote' from project and from Apache? Would > > we > > > >> make > > > >> >>>>>> > > > >> >>>>> a > > > >> >>> > > > >> >>>> new > > > >> >>>> > > > >> >>>>> repo called hbase-connectors and move REST, thrift, and this > new > > > >> patch > > > >> >>>>>> there? > > > >> >>>>>> > > > >> >>>>>> Thanks, > > > >> >>>>>> S > > > >> >>>>>> > > > >> >>>>>> On Mon, Jul 23, 2018 at 3:50 PM Josh Elser < > [email protected]> > > > >> wrote: > > > >> >>>>>> > > > >> >>>>>> I'm -0 for including this into the main hbase tree. I feel > like > > > >> we've > > > >> >>>>>>> made a bit of progress in cleaning up our core, and this > > > strikes me > > > >> >>>>>>> > > > >> >>>>>> as > > > >> >>> > > > >> >>>> a > > > >> >>>> > > > >> >>>>> step in the wrong direction. > > > >> >>>>>>> > > > >> >>>>>>> At the same time, the integration seems nice enough (for the > > > same > > > >> >>>>>>> reasons Andrew points out). Is there a reason this couldn't > > > exist > > > >> >>>>>>> outside of HBase (at the ASF or otherwise)? Given a quick > > > glance at > > > >> >>>>>>> > > > >> >>>>>> the > > > >> >>> > > > >> >>>> patch, it would be quite trivial to keep separate (just > requires > > > some > > > >> >>>>>>> heavier scripting to get it off the ground that the HBase > > > scripts > > > >> do > > > >> >>>>>>> setup for). I feel like that will decrease our debt while we > > > see if > > > >> >>>>>>> people start using it. Our API should be more than stable > > > enough to > > > >> >>>>>>> prevent any worry about drift happening from core to this > > > project. > > > >> >>>>>>> > > > >> >>>>>>> On 7/23/18 6:35 PM, Stack wrote: > > > >> >>>>>>> > > > >> >>>>>>>> We have a very nice contrib sitting up in HBASE-15320 which > > > via a > > > >> >>>>>>>> > > > >> >>>>>>> proxy > > > >> >>>> > > > >> >>>>> -- > > > >> >>>>>>> > > > >> >>>>>>>> so minimal dependencies -- adds source and sink for Kafka > > > Connect. > > > >> >>>>>>>> > > > >> >>>>>>> It > > > >> >>> > > > >> >>>> is > > > >> >>>>>> > > > >> >>>>>>> nicely contained inside two new hbase-kafka-* modules. > > > >> >>>>>>>> > > > >> >>>>>>>> We good w/ taking on this new feature? > > > >> >>>>>>>> > > > >> >>>>>>>> It looks good to me. Check it out up on HBASE-15320. I was > > > going > > > >> to > > > >> >>>>>>>> > > > >> >>>>>>> commit > > > >> >>>>>>> > > > >> >>>>>>>> to tip of branch-2 so it'd show up in hbase-2.2.x unless > you > > > all > > > >> >>>>>>>> > > > >> >>>>>>> want > > > >> >>> > > > >> >>>> some > > > >> >>>>>>> > > > >> >>>>>>>> backporting action going on. > > > >> >>>>>>>> > > > >> >>>>>>>> S > > > >> >>>>>>>> > > > >> >>>>>>>> > > > >> >>>>>>> > > > >> >>>>>> > > > >> >>>>> > > > >> >>>>> -- > > > >> >>>>> Best regards, > > > >> >>>>> Andrew > > > >> >>>>> > > > >> >>>>> Words like orphans lost among the crosstalk, meaning torn from > > > >> truth's > > > >> >>>>> decrepit hands > > > >> >>>>> - A23, Crosstalk > > > >> >>>>> > > > >> >>>>> > > > >> >>>> > > > >> >>> > > > >> >>> > > > >> >> > > > >> > > > > > >
