> HBase does some build magic so they can release against both hadoop1 and > hadoop2 by changing the version to include -hadoopX in the name so they can > different named artifacts for each when they release. So obviously, its > possible for us to do it too :)
It's an ugly hack not magic. We have a script in dev-support, generate-hadoopX-poms.sh, that will munge the versions found in a POM and write a new set of POMs: # Now we do search and replace of explicit strings. The best # way of seeing what the below does is by doing a diff between # the original pom and the generated pom (pom.xml.hadoop1 or # pom.xml.hadoop2). We replace the compat.module variable with # either hbase-hadoop1-compat or hbase-hadoop2-compat, we # replace the version string in all poms, we change modules # to include reference to the non-standard pom name, we # adjust relative paths so child modules can find the parent pom, # and we enable/disable hadoop 1 and hadoop 2 profiles as # appropriate removing a comment string too. We output the # new pom beside the original. We then point Maven at the appropriate munged and version-suffixed root POM, once for hadoop1, once for hadoop2. This practice will thankfully be a thing of the past after HBase 1.0. I don't recommend anyone else adopt it. On Mon, Sep 22, 2014 at 10:19 AM, Jesse Yates <[email protected]> wrote: > From @Flavio > >> I think the jars on maven central are built against hadoop2, so it would >> be REALLY nice to publish both flavours in the next release > > > At the very least, the jars in maven should be what we build by default > (which, AFAIK, is hadoop1). HBase does some build magic so they can release > against both hadoop1 and hadoop2 by changing the version to include -hadoopX > in the name so they can different named artifacts for each when they > release. So obviously, its possible for us to do it too :) > > The question to me is if we should even continue to support hadoop1 in main? > I don't think we should, though we could continue it in the 4.X line for > compatibility's sake. > > ------------------- > Jesse Yates > @jesse_yates > jyates.github.com > > On Mon, Sep 22, 2014 at 9:31 AM, James Taylor <[email protected]> > wrote: >> >> +1 to doing the same for hbase-testing-util. Thanks for the analysis, >> Andrew! >> >> James >> >> On Mon, Sep 22, 2014 at 9:18 AM, Andrew Purtell <[email protected]> >> wrote: >> > On Thu, Sep 18, 2014 at 3:01 PM, James Taylor <[email protected]> >> > wrote: >> >> I see. That makes sense, but it's more of an HBase request than a >> >> Phoenix request. If HBase had a "client-only" pom, then Phoenix could >> >> have a "client-only" pom as well. >> >> >> >> Thanks, >> >> James >> > >> > If you specify only the 'hbase-client' Maven module as the HBase >> > dependency then this pulls in client-only deps (hbase-common, >> > hbase-protocol, commons-codec, commons-collections, commons-io, >> > commons-lang, commons-logging, guava, protobuf-java, netty, zookeeper, >> > htrace-core, jackson-mapper-asl, log4j) >> > >> > I think Phoenix might be pulling in all of the other HBase modules >> > because phoenix-core specifies the 'hbase-testing-util' HBase module >> > as a dependency, and not at test scope: >> > >> > <dependencies> >> > <dependency> >> > <groupId>org.apache.hbase</groupId> >> > <artifactId>hbase-testing-util</artifactId> >> > <exclusions> >> > <exclusion> >> > <groupId>org.jruby</groupId> >> > <artifactId>jruby-complete</artifactId> >> > </exclusion> >> > </exclusions> >> > </dependency> >> > ... >> > >> > hbase-testing-util doesn't contain any code. It is a module you can >> > use that will pull in *everything* needed to start up mini cluster >> > tests, all of the HBase modules including hbase-server and the compat >> > modules, with compile scope. Maven doc says about compile scope: "This >> > is the default scope, used if none is specified. Compile dependencies >> > are available in all classpaths of a project. Furthermore, those >> > dependencies are propagated to dependent projects." >> > >> > Other test dependencies in the phoenix-core POM are included at test >> > scope and tagged as optional, e.g. >> > >> > <dependency> >> > <groupId>org.apache.hadoop</groupId> >> > <artifactId>hadoop-test</artifactId> >> > <optional>true</optional> >> > <scope>test</scope> >> > </dependency> >> > >> > Perhaps the same should be done for hbase-testing-util ? >> > >> > >> >> On Thu, Sep 18, 2014 at 1:52 PM, Flavio Pompermaier >> >> <[email protected]> wrote: >> >>> Because it is not clear which are the exact dependencies to add to a >> >>> pom.xml >> >>> that uses the Phoenix client. >> >>> For example, why do I need to transitively hold all depency of >> >>> hbase-server >> >>> in the client side or both hadoop (1 and 2) compatibility jars?! >> >>> Wouldn't be >> >>> more easy to create a client jar that select only those needed classes >> >>> of >> >>> such jar? >> >>> >> >>> On Sep 18, 2014 9:12 PM, "Mujtaba Chohan" <[email protected]> wrote: >> >>>> >> >>>> Falvio - Client jar is composed on multiple dependency jars extracted >> >>>> in >> >>>> one and is for convenience available in binary download only. This >> >>>> type of >> >>>> bundled jars are not supposed to go in maven repo. as maven >> >>>> automatically >> >>>> resolves required dependencies. To use maven repo., use the >> >>>> phoenix-core as >> >>>> artifact id and maven will pull in all the required dependencies for >> >>>> your >> >>>> project. Then if you want a single jar for your project, you can >> >>>> bundle your >> >>>> project jars in one. >> >>>> >> >>>> <dependency> >> >>>> <groupId>org.apache.phoenix</groupId> >> >>>> <artifactId>phoenix-core</artifactId> >> >>>> <version>4.1.0</version> >> >>>> </dependency> >> >>>> >> >>>> //mujtaba >> >>>> >> >>>> On Thu, Sep 18, 2014 at 11:54 AM, Flavio Pompermaier >> >>>> <[email protected]> wrote: >> >>>>> >> >>>>> Is the client jar available on maven central? >> >>>>> >> >>>>> On Sep 18, 2014 8:46 PM, "anil gupta" <[email protected]> wrote: >> >>>>>> >> >>>>>> Hi Flavio, >> >>>>>> >> >>>>>> Phoenix-core is for sever side. If this app is for client side then >> >>>>>> you >> >>>>>> can use client jars. Maybe you can try to use >> >>>>>> phoenix-client-without-hbase.jar. In this case, i think you will at >> >>>>>> least >> >>>>>> need hbase-client jar in your client app to be added in your pom. >> >>>>>> >> >>>>>> Thanks, >> >>>>>> Anil Gupta >> >>>>>> >> >>>>>> On Thu, Sep 18, 2014 at 6:13 AM, Flavio Pompermaier >> >>>>>> <[email protected]> wrote: >> >>>>>>> >> >>>>>>> Is it really necessary the dependency to hbase-server..? Then if I >> >>>>>>> try >> >>>>>>> to run my application from a webapp using resteasy I get this >> >>>>>>> error: >> >>>>>>> >> >>>>>>> Caused by: >> >>>>>>> org.jboss.as.server.deployment.DeploymentUnitProcessingException: >> >>>>>>> JBAS011232: Only one JAX-RS Application Class allowed. >> >>>>>>> com.sun.jersey.api.core.ClasspathResourceConfig >> >>>>>>> com.sun.jersey.server.impl.application.DeferredResourceConfig >> >>>>>>> com.sun.jersey.api.core.ScanningResourceConfig >> >>>>>>> org.apache.hadoop.hbase.rest.ResourceConfig >> >>>>>>> com.sun.jersey.api.core.ClassNamesResourceConfig >> >>>>>>> com.sun.jersey.api.core.ApplicationAdapter >> >>>>>>> com.sun.jersey.api.core.WebAppResourceConfig >> >>>>>>> com.sun.jersey.api.core.PackagesResourceConfig >> >>>>>>> com.sun.jersey.api.core.DefaultResourceConfig >> >>>>>>> com.sun.jersey.api.core.ResourceConfig >> >>>>>>> at >> >>>>>>> >> >>>>>>> org.jboss.as.jaxrs.deployment.JaxrsScanningProcessor.scan(JaxrsScanningProcessor.java:209) >> >>>>>>> >> >>>>>>> On Thu, Sep 18, 2014 at 11:40 AM, Flavio Pompermaier >> >>>>>>> <[email protected]> wrote: >> >>>>>>>> >> >>>>>>>> I was able to obtain a minimal set of dependencies *for hadoop1) >> >>>>>>>> using: >> >>>>>>>> <dependency> >> >>>>>>>> <groupId>org.apache.phoenix</groupId> >> >>>>>>>> <artifactId>phoenix</artifactId> >> >>>>>>>> <version>4.1.0</version> >> >>>>>>>> </dependency> >> >>>>>>>> <dependency> >> >>>>>>>> <groupId>org.apache.hbase</groupId> >> >>>>>>>> <artifactId>hbase-client</artifactId> >> >>>>>>>> <version>0.98.4-hadoop1</version> >> >>>>>>>> </dependency> >> >>>>>>>> <!-- Need hbase server just for >> >>>>>>>> ConstantSizeRegionSplitPolicy class.. --> >> >>>>>>>> <dependency> >> >>>>>>>> <groupId>org.apache.hbase</groupId> >> >>>>>>>> <artifactId>hbase-server</artifactId> >> >>>>>>>> <version>0.98.4-hadoop1</version> >> >>>>>>>> <exclusions> >> >>>>>>>> <exclusion> >> >>>>>>>> <artifactId>*</artifactId> >> >>>>>>>> <groupId>*</groupId> >> >>>>>>>> </exclusion> >> >>>>>>>> </exclusions> >> >>>>>>>> </dependency> >> >>>>>>>> <dependency> >> >>>>>>>> <groupId>org.apache.phoenix</groupId> >> >>>>>>>> <artifactId>phoenix-hadoop-compat</artifactId> >> >>>>>>>> <version>4.1.0</version> >> >>>>>>>> <exclusions> >> >>>>>>>> <!-- conflict with >> >>>>>>>> hbase-client-0.98.4-hadoop1 version --> >> >>>>>>>> <exclusion> >> >>>>>>>> <artifactId>hbase-common</artifactId> >> >>>>>>>> <groupId>org.apache.hbase</groupId> >> >>>>>>>> </exclusion> >> >>>>>>>> </exclusions> >> >>>>>>>> </dependency> >> >>>>>>>> >> >>>>>>>> but I still get this exception (that however seems not causing >> >>>>>>>> any >> >>>>>>>> issue): >> >>>>>>>> java.lang.RuntimeException: Could not create interface >> >>>>>>>> org.apache.phoenix.trace.PhoenixSpanReceiver Is the hadoop >> >>>>>>>> compatibility jar >> >>>>>>>> on the classpath? >> >>>>>>>> at >> >>>>>>>> >> >>>>>>>> org.apache.hadoop.hbase.CompatibilityFactory.getInstance(CompatibilityFactory.java:60) >> >>>>>>>> at >> >>>>>>>> >> >>>>>>>> org.apache.phoenix.trace.TracingCompat.newTraceMetricSource(TracingCompat.java:40) >> >>>>>>>> at >> >>>>>>>> >> >>>>>>>> org.apache.phoenix.trace.util.Tracing.addTraceMetricsSource(Tracing.java:294) >> >>>>>>>> at >> >>>>>>>> >> >>>>>>>> org.apache.phoenix.jdbc.PhoenixConnection.<clinit>(PhoenixConnection.java:125) >> >>>>>>>> at >> >>>>>>>> >> >>>>>>>> org.apache.phoenix.query.ConnectionQueryServicesImpl$9.call(ConnectionQueryServicesImpl.java:1516) >> >>>>>>>> at >> >>>>>>>> >> >>>>>>>> org.apache.phoenix.query.ConnectionQueryServicesImpl$9.call(ConnectionQueryServicesImpl.java:1489) >> >>>>>>>> at >> >>>>>>>> >> >>>>>>>> org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:77) >> >>>>>>>> at >> >>>>>>>> >> >>>>>>>> org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:1489) >> >>>>>>>> at >> >>>>>>>> >> >>>>>>>> org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:162) >> >>>>>>>> at >> >>>>>>>> >> >>>>>>>> org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.connect(PhoenixEmbeddedDriver.java:129) >> >>>>>>>> at >> >>>>>>>> >> >>>>>>>> org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:133) >> >>>>>>>> at java.sql.DriverManager.getConnection(DriverManager.java:571) >> >>>>>>>> at java.sql.DriverManager.getConnection(DriverManager.java:233) >> >>>>>>>> Caused by: java.util.NoSuchElementException >> >>>>>>>> at >> >>>>>>>> java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:357) >> >>>>>>>> at java.util.ServiceLoader$1.next(ServiceLoader.java:445) >> >>>>>>>> at >> >>>>>>>> >> >>>>>>>> org.apache.hadoop.hbase.CompatibilityFactory.getInstance(CompatibilityFactory.java:46) >> >>>>>>>> ... 14 more >> >>>>>>>> >> >>>>>>>> How can I fix those problems? >> >>>>>>>> >> >>>>>>>> On Thu, Sep 18, 2014 at 10:34 AM, Flavio Pompermaier >> >>>>>>>> <[email protected]> wrote: >> >>>>>>>>> >> >>>>>>>>> Hi to all, >> >>>>>>>>> >> >>>>>>>>> I'm trying to write a J2EE application that uses the Phoenix >> >>>>>>>>> JDBC >> >>>>>>>>> driver for HBase 0.98 (not on yarn for tests but it will be in >> >>>>>>>>> production) >> >>>>>>>>> through maven. >> >>>>>>>>> Do I really need all the jars of the hbase-server side contained >> >>>>>>>>> in >> >>>>>>>>> the phoenix core jar..? >> >>>>>>>>> I can't see a phoenix-client jar on the central maven repository >> >>>>>>>>> while in the documentation there's to use that jar! >> >>>>>>>>> >> >>>>>>>>> Thanks in advance, >> >>>>>>>>> Flavio >> >>>>>>>> >> >>>>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> -- >> >>>>>> Thanks & Regards, >> >>>>>> Anil Gupta >> >>>> >> >>>> >> >>> >> > >> > >> > >> > -- >> > Best regards, >> > >> > - Andy >> > >> > Problems worthy of attack prove their worth by hitting back. - Piet >> > Hein (via Tom White) > > -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
