The commands look good. However, for some reason the Phoenix build command doesn't pick up the rebuilt artifacts. You need to run your maven processes with debug enabled (-X), and check your local maven repo to see what goes wrong.
On Fri, Nov 19, 2021 at 12:22 PM Mark Jens <[email protected]> wrote: > Hi Istvan, > > I've patched HBase 2.4.8 with the same changes as in > https://github.com/apache/hbase/pull/3860. > hbase-2.4.8$ mvn install -Dhadoop.profile=3.0 -DskipTests passed > successfully! > > Following BUILDING.md I installed Phoenix with: mvn clean install > -Dhbase.profile=2.4 -Dhbase.version=2.4.8 -DskipTests > And finally tried to test it with: mvn verify -Dhbase.profile=2.4 > -Dhbase.version=2.4.8 but again it failed with > > java.lang.IncompatibleClassChangeError: Found interface > org.apache.hadoop.hdfs.protocol.HdfsFileStatus, but class was expected > at > org.apache.hadoop.hbase.io > .asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:536) > at > org.apache.hadoop.hbase.io > .asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$400(FanOutOneBlockAsyncDFSOutputHelper.java:112) > at > org.apache.hadoop.hbase.io > .asyncfs.FanOutOneBlockAsyncDFSOutputHelper$8.doCall(FanOutOneBlockAsyncDFSOutputHelper.java:616) > at > org.apache.hadoop.hbase.io > .asyncfs.FanOutOneBlockAsyncDFSOutputHelper$8.doCall(FanOutOneBlockAsyncDFSOutputHelper.java:611) > at > > org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) > at > org.apache.hadoop.hbase.io > .asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:624) > at > org.apache.hadoop.hbase.io > .asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:53) > at > > org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:180) > at > > org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:166) > at > > org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:113) > at > > org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:669) > at > > org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:130) > at > > org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:841) > at > > org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:548) > at > > org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:489) > at > > org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:160) > at > > org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62) > at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:296) > at > > org.apache.phoenix.hbase.index.write.recovery.TestPerRegionIndexWriteCache.setUp(TestPerRegionIndexWriteCache.java:109) > ... > > What I am doing wrong ? > > Thanks! > > Mark > > > > On Thu, 18 Nov 2021 at 11:16, Mark Jens <[email protected]> wrote: > > > Hi, > > > > On Thu, 18 Nov 2021 at 10:38, Istvan Toth <[email protected]> > > wrote: > > > >> Phoenix probably won't work with HBase 3.x, but looking at the linked > >> commit, it should be fairly straightforward to apply that to HBase 2.4. > >> > >> I'm not sure why that hasn't been backported to 2.x, perhaps HBase > doesn't > >> have test infra set up for ARM. > >> You may want to discuss backporting that change to the active Hbase 2.x > >> branches with the HBase project. > >> > > > > I've asked for a backport at > > > https://issues.apache.org/jira/browse/HBASE-23612?focusedCommentId=17445761&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17445761 > > > > Apologies for creating > https://issues.apache.org/jira/browse/PHOENIX-6595. > > Initially I thought that Phoenix can apply the patch since it already > > requires manual build of HBase. > > But you are right - it would be much better if the improvement is > > backported to HBase 2.x ! > > > > > >> > >> > >> On Wed, Nov 17, 2021 at 12:35 PM Mark Jens <[email protected]> > wrote: > >> > >> > Hi Istvan, > >> > > >> > It seems HBase support for ARM64 is available only in 3.x: > >> > > >> > > >> > https://github.com/apache/hbase/commit/5480493f5f7b01b496f54215334543f2a82c6ba7 > >> > Would Phoenix work with HBase 3.x ? > >> > > >> > On Wed, 17 Nov 2021 at 13:28, Mark Jens <[email protected]> > wrote: > >> > > >> > > Thanks for the hint! > >> > > > >> > > Unfortunately HBase-2.4.8 build fails with: > >> > > > >> > > INFO] BUILD FAILURE > >> > > [INFO] > >> > > > >> ------------------------------------------------------------------------ > >> > > [INFO] Total time: 01:42 min > >> > > [INFO] Finished at: 2021-11-17T11:23:19Z > >> > > [INFO] > >> > > > >> ------------------------------------------------------------------------ > >> > > [ERROR] Failed to execute goal > >> > > org.xolstice.maven.plugins:protobuf-maven-plugin:0.6.1:compile > >> > > (compile-protoc) on project hbase-protocol: Unable to resolve > >> artifact: > >> > > Missing: > >> > > [ERROR] ---------- > >> > > [ERROR] 1) com.google.protobuf:protoc:exe:linux-aarch_64:2.5.0 > >> > > [ERROR] > >> > > [ERROR] Try downloading the file manually from the project > website. > >> > > [ERROR] > >> > > [ERROR] Then, install it using the command: > >> > > [ERROR] mvn install:install-file -DgroupId=com.google.protobuf > >> > > -DartifactId=protoc -Dversion=2.5.0 -Dclassifier=linux-aarch_64 > >> > > -Dpackaging=exe -Dfile=/path/to/file > >> > > [ERROR] > >> > > [ERROR] Alternatively, if you host your own repository you can > >> deploy > >> > > the file there: > >> > > [ERROR] mvn deploy:deploy-file -DgroupId=com.google.protobuf > >> > > -DartifactId=protoc -Dversion=2.5.0 -Dclassifier=linux-aarch_64 > >> > > -Dpackaging=exe -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id] > >> > > [ERROR] > >> > > [ERROR] Path to dependency: > >> > > [ERROR] 1) org.apache.hbase:hbase-protocol:jar:2.4.8 > >> > > [ERROR] 2) com.google.protobuf:protoc:exe:linux-aarch_64:2.5.0 > >> > > > >> > > > >> > > > >> > > > >> > > On Wed, 17 Nov 2021 at 12:27, Istvan Toth > <[email protected] > >> > > >> > > wrote: > >> > > > >> > >> You need to recompile HBase. > >> > >> See BULIDING.md > >> > >> > >> > >> On Wed, Nov 17, 2021 at 10:33 AM Mark Jens <[email protected]> > >> > wrote: > >> > >> > >> > >> > Hello, > >> > >> > > >> > >> > I am trying to build Phoenix on Ubuntu 20.04.3 ARM64. > >> > >> > > >> > >> > Phoenix Core module fails with: > >> > >> > > >> > >> > [ERROR] > >> > >> > > >> > >> > > >> > >> > >> > > >> > org.apache.phoenix.hbase.index.write.recovery.TestPerRegionIndexWriteCache.testMultipleAddsForSingleRegion > >> > >> > Time elapsed: 0.025 s <<< ERROR! > >> > >> > java.lang.IncompatibleClassChangeError: Found interface > >> > >> > org.apache.hadoop.hdfs.protocol.HdfsFileStatus, but class was > >> expected > >> > >> > at > >> > >> > org.apache.hadoop.hbase.io > >> > >> > > >> > >> > >> > > >> > .asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:536) > >> > >> > at > >> > >> > org.apache.hadoop.hbase.io > >> > >> > > >> > >> > >> > > >> > .asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$400(FanOutOneBlockAsyncDFSOutputHelper.java:112) > >> > >> > at > >> > >> > org.apache.hadoop.hbase.io > >> > >> > > >> > >> > >> > > >> > .asyncfs.FanOutOneBlockAsyncDFSOutputHelper$8.doCall(FanOutOneBlockAsyncDFSOutputHelper.java:616) > >> > >> > at > >> > >> > org.apache.hadoop.hbase.io > >> > >> > > >> > >> > >> > > >> > .asyncfs.FanOutOneBlockAsyncDFSOutputHelper$8.doCall(FanOutOneBlockAsyncDFSOutputHelper.java:611) > >> > >> > at > >> > >> > > >> > >> > > >> > >> > >> > > >> > org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) > >> > >> > at > >> > >> > org.apache.hadoop.hbase.io > >> > >> > > >> > >> > >> > > >> > .asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:624) > >> > >> > at > >> > >> > org.apache.hadoop.hbase.io > >> > >> > > >> .asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:53) > >> > >> > at > >> > >> > > >> > >> > > >> > >> > >> > > >> > org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:180) > >> > >> > at > >> > >> > > >> > >> > > >> > >> > >> > > >> > org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:166) > >> > >> > at > >> > >> > > >> > >> > > >> > >> > >> > > >> > org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:113) > >> > >> > at > >> > >> > > >> > >> > > >> > >> > >> > > >> > org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:669) > >> > >> > at > >> > >> > > >> > >> > > >> > >> > >> > > >> > org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:130) > >> > >> > at > >> > >> > > >> > >> > > >> > >> > >> > > >> > org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:841) > >> > >> > at > >> > >> > > >> > >> > > >> > >> > >> > > >> > org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:548) > >> > >> > at > >> > >> > > >> > >> > > >> > >> > >> > > >> > org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:489) > >> > >> > at > >> > >> > > >> > >> > > >> > >> > >> > > >> > org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:160) > >> > >> > at > >> > >> > > >> > >> > > >> > >> > >> > > >> > org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62) > >> > >> > at > >> org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:296) > >> > >> > at > >> > >> > > >> > >> > > >> > >> > >> > > >> > org.apache.phoenix.hbase.index.write.recovery.TestPerRegionIndexWriteCache.setUp(TestPerRegionIndexWriteCache.java:109) > >> > >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >> > >> > .... > >> > >> > [INFO] > >> > >> > [INFO] Results: > >> > >> > [INFO] > >> > >> > [ERROR] Errors: > >> > >> > [ERROR] TestPerRegionIndexWriteCache.setUp:109 » > >> > >> IncompatibleClassChange > >> > >> > Found interfa... > >> > >> > [ERROR] TestPerRegionIndexWriteCache.setUp:109 » > >> > >> IncompatibleClassChange > >> > >> > Found interfa... > >> > >> > [ERROR] TestPerRegionIndexWriteCache.setUp:109 » > >> > >> IncompatibleClassChange > >> > >> > Found interfa... > >> > >> > [INFO] > >> > >> > [ERROR] Tests run: 1909, Failures: 0, Errors: 3, Skipped: 6 > >> > >> > [INFO] > >> > >> > [INFO] > >> > >> > > >> > > ------------------------------------------------------------------------ > >> > >> > [INFO] Reactor Summary for Apache Phoenix 5.2.0-SNAPSHOT: > >> > >> > [INFO] > >> > >> > [INFO] Apache Phoenix ..................................... > >> SUCCESS [ > >> > >> > 2.034 s] > >> > >> > [INFO] Phoenix Hbase 2.4.1 compatibility .................. > >> SUCCESS [ > >> > >> > 5.358 s] > >> > >> > [INFO] Phoenix Hbase 2.4.0 compatibility .................. > >> SUCCESS [ > >> > >> > 3.946 s] > >> > >> > [INFO] Phoenix Hbase 2.3.0 compatibility .................. > >> SUCCESS [ > >> > >> > 4.437 s] > >> > >> > [INFO] Phoenix Hbase 2.2.5 compatibility .................. > >> SUCCESS [ > >> > >> > 4.004 s] > >> > >> > [INFO] Phoenix Hbase 2.1.6 compatibility .................. > >> SUCCESS [ > >> > >> > 3.966 s] > >> > >> > [INFO] Phoenix Core ....................................... > FAILURE > >> > >> [01:25 > >> > >> > min] > >> > >> > [INFO] Phoenix - Pherf .................................... > SKIPPED > >> > >> > ... > >> > >> > > >> > >> > Any idea why this breaks ? > >> > >> > > >> > >> > It does not look ARM64 specific to me. I will try on x64 too. > >> > >> > > >> > >> > Thanks! > >> > >> > > >> > >> > Mark > >> > >> > > >> > >> > >> > >> > >> > >> -- > >> > >> *István Tóth* | Staff Software Engineer > >> > >> [email protected] <https://www.cloudera.com> > >> > >> [image: Cloudera] <https://www.cloudera.com/> > >> > >> [image: Cloudera on Twitter] <https://twitter.com/cloudera> > [image: > >> > >> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: > >> > >> Cloudera > >> > >> on LinkedIn] <https://www.linkedin.com/company/cloudera> > >> > >> <https://www.cloudera.com/> > >> > >> ------------------------------ > >> > >> > >> > > > >> > > >> > >> > >> -- > >> *István Tóth* | Staff Software Engineer > >> [email protected] <https://www.cloudera.com> > >> [image: Cloudera] <https://www.cloudera.com/> > >> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: > >> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: > >> Cloudera > >> on LinkedIn] <https://www.linkedin.com/company/cloudera> > >> <https://www.cloudera.com/> > >> ------------------------------ > >> > > >
