Hi Pulkit, The steps for building a HBase-2.x branch against Hadoop3 are documented here [0].
In this case, you should omit `-Dhadoop-two.version=3.1.2 -Phadoop-3.0` and instead use `-Dhadoop.profile=3.0`. Activate this profile and the `hadoop-three.version` property should be accepted. I also suggest that you first package and install before running tests.You may also want to run maven with several parallel threads, via --threads. This will depend a lot on your build hardware. In total, I suggest: $ mvn clean install -DskipTests -Dhadoop.profile=3.0 -Dhadoop-three.version=3.1.2 $ mvn test -Dhadoop.profile=3.0 -Dhadoop-three.version=3.1.2 Note also that our test suite is quite extensive, and perhaps only a subset of it is of interest to you. Have a look at [1] to see how it's broken down. You may want to only test a subset of the modules, and only a subset of the test classifications. FYI, there have been chronic test failures in several modules that make use of the Hadoop MiniCluster feature for some time. This includes hbase-mapreduce and hbase-server. The situation was improved for branch-2.3, and further for branch-2.4, but this is an ongoing challenge and will vary quite a lot based on the capability of your test machine and your luck when you run. I also advise disconnecting from any VPN, as well as logging out of any kerberos principals, on the machine that runs the tests. Good Luck, Nick [0]: http://hbase.apache.org/book.html#maven.build.hadoop [1]: http://hbase.apache.org/book.html#hbase.unittests.cmds On Thu, Apr 15, 2021 at 6:53 AM Pulkit Chawla <[email protected]> wrote: > > > From: Pulkit Chawla <[email protected]> > Date: Thursday, 15 April 2021 at 5:12 AM > To: [email protected] <[email protected]> > Subject: Building hbase 2.2.6 with Hadoop 3.1.2 > Hi, > I am trying to build and test hbase-2.2.6 with my prebuilt Hadoop-3.1.2. > > For that, I used the following arguments with maven:: > > Mvn package -Dhadoop-three.version=3.1.2 -Dhadoop-two.version=3.1.2 > -Phadoop-3.0 > > Since I want to only use my prebuilt hadoop3.1.2, I am giving > -Dhadoop-two.version=3.1.2, so that for dependencies it would not try to > pick anything from Hadoop-2.8.5 (which is the default hadoop2 version in > pom) and use only 3.1.2 one . > > Problem is unit tests are failing at hbase-mapreduce module. Do > hbase-mapreduce module has a hard dependency on hadoop2?? > If not how can I make it work with only hadoop3.1.2 ?? > > It would really help a lot if anyone can answer my query. > > Thanks and regards, > Pulkit >
