All, To what extent does the Accumulo Client rely on the Hadoop Client? I apologize if the question is a bit obtuse. But I got into dependency weeds trying to get the Hadoop Client to work in OSGI. (See below Hadoop Client woes) I am now wondering if I OSGified Accumulo's client would I encounter the same-old-same-old or somehow dodge the bullet.
Would anyone else be interested? <Hadoop Client woes> I sat down to do what Corey suggested and took a shot at getting the Hadoop Client working in OSGi. I used the service mix bundles, but alas, it seems that somewhere in the dependencies something wants to use JAAS and that is stopping the show. I creating a fragment that exports the required package so OSGI can find them--nothing doing; I'm stuck. If one Googles, one finds JAAS is problematic in OSGi as are a number of J2EE technologies. </Hadoop Client woes> On Tue, Apr 1, 2014 at 11:22 AM, Geoffry Roberts <threadedb...@gmail.com>wrote: > Thank you Corey, > > I was unaware of the service mix Hadoop client. It's funny that no one on > the Hadoop list ever mentioned it. > > You say you have 1.4 working in OSGi. Did you do a proper port or just > wrap it with something like bnd? I have Hadoop 2.3.0 so I need to use > Accumulo 1.5.1. I'm glad to hear the bane of split packages has been put > asunder. > > I am using equinox for now. Most of my legacy code in based on bndtools > and I exercise it in equinox. I've used equinox in the past for production > and things went well. > > I just grabbed the Hadoop core & client from service mix then zookeeper. > At the least, they were accepted into my bndtools repository as valid > bundles. I noticed that a number of the usual Hadoop dependencies are in > service mix as well so I sense a glimmer of hope. Wrt Accumulo, I think > I'll take a stab at either wrapping or porting core, frame, start, and > trace to OSGi and see how much trouble I get into. > > It's either do the above or abandon OSGi altogether for this project. > > My objective is to persist EMF object graphs into Accumulo. These graphs > were built by others and are based on ISO standards so I need to walk the > straight and narrow and not drop a stitch. I have code that does what I > need (persist any graph sight unseen) into MongoDB. I need to adapt said > code to Accumulo. All the above is OSGi based so it would really help if I > can keep the Accumulo end of things in OSGi as well. > > Wish me well > > > On Mon, Mar 31, 2014 at 8:34 PM, Corey Nolet <cjno...@gmail.com> wrote: > >> Geoffry, >> >> What OSGi container are you using currently? The servicemix Hadoop bundle >> should get you going with the Hadoop client dependencies at least [1]. It >> looks like one of the servicemix guys created a Hadoop ticket for making >> bundles of their jars as well [2], though it doesn't look like there's been >> any movement on it. >> >> I recently had to get the CDH3u4 client code working in Karaf. A good >> starting place for me was [3], however I did need to make updates to >> versions of many of the dependencies to get it functioning as expected. [3] >> will get you at least started with dependent bundles and the proper >> imports/exports to get it working. >> >> I've got the Accumulo client running in OSGi. If I recall correctly, >> versions 1.4 and above do not split packages across jars so it's really >> just a matter of getting the dependencies right. Zookeeper also ships as a >> bundle [4]. >> >> >> Hope this helps. >> >> [1] >> http://mvnrepository.com/artifact/org.apache.servicemix.bundles/org.apache.servicemix.bundles.hadoop-core/ >> [2] https://issues.apache.org/jira/browse/HADOOP-8446 >> [3] https://github.com/jbonofre/karaf-hadoop >> [4] https://issues.apache.org/jira/browse/ZOOKEEPER-425 >> >> >> >> On Mon, Mar 31, 2014 at 11:37 AM, Geoffry Roberts <threadedb...@gmail.com >> > wrote: >> >>> Luk, >>> >>> Thanks for the link, but I am a bit lost. wso2 offers middleware, >>> apparently you believe this will help my situation. If it's not too much, >>> can you expand? >>> >>> >>> On Mon, Mar 31, 2014 at 11:13 AM, Luk Vervenne <l...@synergetics.be>wrote: >>> >>>> osgi... see wso2.com >>>> >>>> On 31 Mar 2014, at 16:58, Geoffry Roberts <threadedb...@gmail.com> >>>> wrote: >>>> >>>> > All, >>>> > >>>> > I have a project for which Accumulo it appears will serve well. >>>> However, I have a significant amount of code I want to leverage that runs >>>> in OSGi. I don't need for Accumulo itself to be OSGi based but the >>>> Accumulo client yes. I see that the Accumulo client uses all the >>>> dependencies of the Hadoop client and therefore is not OSGi ready at this >>>> time. The Hadoop client certainly doesn't do OSGi--I don't think it can >>>> even spell it :-)--and attempting to make it so starts turning into a sure >>>> path to a long sojourn through dependency hell. I know, I've tried. >>>> > >>>> > Nonetheless, I would like to ask: Is there any interest in the >>>> Accumulo world of having an OSGi based client for this otherwise very >>>> appealing database? >>>> > >>>> > Thanks mucho >>>> > -- >>>> > There are ways and there are ways, >>>> > >>>> > Geoffry Roberts >>>> >>>> >>> >>> >>> -- >>> There are ways and there are ways, >>> >>> Geoffry Roberts >>> >> >> > > > -- > There are ways and there are ways, > > Geoffry Roberts > -- There are ways and there are ways, Geoffry Roberts
