If there are no BigTop-specific maven names or repositories, then:
* Are we expected to build BigTop from source code?
* Does building BigTop from source code deploy hadoop artifact to our local
maven repository?
Building HBase package by using Bigtop installs artifacts to your local
repository.
For the example of hbase-common-2.4.13, the pom.xml will be same to the one
available on public repositories since Bigtop does not apply patches to
the pom.xml of hbase-common.
The build-time options used by Bigtop like following does not affect the
installed pom.xml::
$ mvn -Phadoop-3.0 -Dhadoop-three.version=3.3.4
-Dhadoop.guava.version=27.0-jre -Djetty.version=9.3.29.v20201019
-Dzookeeper.version=3.5.9 -DskipTests -Dcheckstyle.skip=true
-Dadditionalparam=-Xdoclint:none clean install
If your product depends on hbase-common-2.4.13, I guess adding
Maven opts like '-Dhadoop.profile=3.0' will pull hadoop-3.1.2 (instead of
2.10.0)
as the transitive dependencies of hbase-common.
- https://github.com/apache/hbase/blob/rel/2.4.13/hbase-common/pom.xml
- https://github.com/apache/hbase/blob/rel/2.4.13/pom.xml#L544
If your product depends on hadoop-common too and
you want to build it against hadoop-common-3.3.4,
it will be pulled by adding explicit dependency
as a result of dependency convergence.
On 2023/02/03 7:52, Baba Open wrote:
posted this question on user group - no sure if it was the right place -
posting it here (hope that is ok).
Sorry I do not know much about BigTop, just getting started. We have a
Java (maven) project that depends on Hadoop & HBase. We are targeting
Bigtop 3.2.0. We have updated our maven pom file to point to the versions
as specified in:
https://github.com/apache/bigtop/blob/release-3.2.0/bigtop.bom
Not sure if this is correct, but we suspect that BigTop is building HBase
2.4.13 with hadoop3 profile. Are there any maven specific monikers (or
repository) we can use to access the binaries used by BigTop?
For example, when we use
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-common</artifactId>
<version>2.4.13</version>
</dependency>
Maven just downloads the generic hbase-common jar file (which relies on
Hadoop 2.10).
Another example, to help clarify the question further. If we were
targeting a cloudera cluster for example, we would be changing our maven
dependency to
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-common</artifactId>
<version>x.y.z_cdh-a.b.c...</version>
</dependency>
If there are no BigTop-specific maven names or repositories, then:
* Are we expected to build BigTop from source code?
* Does building BigTop from source code deploy hadoop artifact to our local
maven repository?
* How do we avoid clashes? For example, how do we maintain the generic
version of hbase-common 2.4.13 and the BigTop version in our repos?
* I couldn't find anything in the documentation that would help answer the
above questions but perhaps I missed it.
Thanks in advance
