Thanks Ryan, I havent tested hbase-writer with hbase 0.20.0 yet. Didnt know that was a client dependency. I thought you only need zookeeper on the server side. Once hbase 0.20.0 is released, ill be testing it out. -Ryan
On Thu, Jul 30, 2009 at 1:56 PM, Ryan Rawson <[email protected]> wrote: > you will also require zookeeper as well. and the config files of course. > > -ryan > > On Thu, Jul 30, 2009 at 10:53 AM, Ryan Smith<[email protected]> > wrote: > > Maybe someone can correct me if I'm wrong, but if your application is > > accessing hbase as a client, you only need log4j, hadoop-core and hbase. > It > > would be nice to have hadoop/hbase build under maven2, but in my opinion, > it > > would have to be a switch from their current build system to make it > worth > > while. For example, one hbase source package references thrfit, but not > > everyone uses thrift, so ideally in a maven2 pom.xml, you want to create > > modules, hbase-core, hbase-thrift, hbase-zookeeper, etc. Seems like at > that > > point, you would want to switch the whole project over to maven so youre > not > > doing double work for build maintenence. For now, referencing what jars > you > > need from your app's pom.xml seems to make the most sense for > hadoop/hbase - > > maybe not for other projects with large amounts of runtime dependencies > > though. Here's hbase-writer's pom.xml : > > > > http://code.google.com/p/hbase-writer/source/browse/trunk/pom.xml > > > > If you scroll down to the dependencies, you'll see them all there. The > > application only uses hbase to create and write records to hbase tables > so > > all it depends on is hadoop jar, hbase jar and log4j jar. All other deps > > are for heritrix and one for testng for unit tests. > > Dont let me discourage you from creating a jira ticket and submitting a > > patch to create a pom.xml for hbase though! :) I just dont find it a > good > > use of my own time to maintain one yet. > > > > -Ryan > > > > > > On Thu, Jul 30, 2009 at 12:24 PM, Fernando Padilla <[email protected] > >wrote: > > > >> I was thinking you would create a pom.xml for hadoop/hbase, that would > >> include all of the transitive dependencies.. so that way you only had to > >> depend on hadoop or hbase, and all their dependencies would be brought > in. > >> > >> I guess it's easier to maintain without a pom.xml, but not easier to > use.. > >> since everyone would have to track down the dependencies and add them to > >> their pom.xml as well. :) > >> > >> But, baby steps :) > >> > >> > >> > >> On 7/30/09 9:10 AM, Ryan Smith wrote: > >> > >>> Fernando, > >>> > >>> Thanks for the link to free nexus hosting. I appreciate that. I didnt > >>> know > >>> that maven central repo would pick up jars from other repos like that. > >>> I'll > >>> have to check that out. > >>> As for your questions, I think you have a misunderstanding. This is a > >>> nexus > >>> repository so i can build hbase-writer using maven2. > >>> http://code.google.com/p/hbase-writer > >>> > >>> So you're maintaining your own pom.xml for hadoop and hbase? > >>>> > >>> No, I am not maintaining a pom.xml for hadoop and hbase because I am > not > >>> building hadoop or hbase using maven2. You dont need a pom.xml to use > a > >>> jar > >>> as a dependency in maven2. Just deploy the hbase/hadoop jars to the > >>> repository, then in your project pom.xml, reference hadoop and hbase as > >>> dependencies. No pom maintenence required. > >>> > >>> I would create a pom.xml for hbase if they wanted to switch over to > >>> maven2, > >>> but I think the gang is fine with the build system as it is. ;) > >>> > >>> -Ryan > >>> > >>> > >>> > >>> On Thu, Jul 30, 2009 at 11:35 AM, Fernando Padilla<[email protected] > >>> >wrote: > >>> > >>> That's great. Thank you. > >>>> > >>>> So you're maintaining your own pom.xml for hadoop and hbase? > Could/Should > >>>> that be in the svn somewhere? > >>>> > >>>> > >>>> If you are a nexus user, you might be interested to know that Nexus > >>>> offers > >>>> a Nexus server for OSS projects (like hadoop and hbase): > >>>> > >>>> https://docs.sonatype.com/display/NX/OSS+Repository+Hosting > >>>> > >>>> So I would encourage you to volunteer to help set this up for hadoop > and > >>>> hbase. :) That way random people won't use up your bandwidth, and/or > the > >>>> artifacts will start showing up in the central repo.. much easier to > deal > >>>> with. :) > >>>> > >>>> What do you think? > >>>> > >>>> > >>>> > >>>> On 7/30/09 5:20 AM, Ryan Smith wrote: > >>>> > >>>> I keep hadoop& hbase jars in a maven2 repo for hbase-writer > project: > >>>>> > >>>>> http://repo1.opensourcemasters.org:8081/nexus/ > >>>>> > >>>>> You are free to reference this for now until hadoop& hbase are in > the > >>>>> official maven repo. > >>>>> Ill be adding 0.20.0 once hbase has a final release. > >>>>> > >>>>> -Ryan > >>>>> > >>>>> > >>>>> On Wed, Jul 29, 2009 at 10:50 PM, Ryan Rawson<[email protected]> > >>>>> wrote: > >>>>> > >>>>> No sir. > >>>>> > >>>>>> There was some talk of adopting the ivy build system that hadoop > has, > >>>>>> but its fairly non-trivial, and we had plenty of other things to do. > >>>>>> > >>>>>> If one was to show up at the hackathon next week and whack at it, it > >>>>>> might have a very good chance of adoption :-) > >>>>>> > >>>>>> -ryan > >>>>>> > >>>>>> On Wed, Jul 29, 2009 at 7:45 PM, Fernando Padilla<[email protected] > > > >>>>>> wrote: > >>>>>> > >>>>>> So.. are hbase libraries on some maven repository somewhere?? So I > can > >>>>>>> simply depend on it? > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>> > > >
