Hi Istvan, I've patched HBase 2.4.8 with the same changes as in https://github.com/apache/hbase/pull/3860. hbase-2.4.8$ mvn install -Dhadoop.profile=3.0 -DskipTests passed successfully!
Following BUILDING.md I installed Phoenix with: mvn clean install -Dhbase.profile=2.4 -Dhbase.version=2.4.8 -DskipTests And finally tried to test it with: mvn verify -Dhbase.profile=2.4 -Dhbase.version=2.4.8 but again it failed with java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.hdfs.protocol.HdfsFileStatus, but class was expected at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:536) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$400(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$8.doCall(FanOutOneBlockAsyncDFSOutputHelper.java:616) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$8.doCall(FanOutOneBlockAsyncDFSOutputHelper.java:611) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:624) at org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:53) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:180) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:166) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:113) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:669) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:130) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:841) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:548) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:489) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:160) at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62) at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:296) at org.apache.phoenix.hbase.index.write.recovery.TestPerRegionIndexWriteCache.setUp(TestPerRegionIndexWriteCache.java:109) ... What I am doing wrong ? Thanks! Mark On Thu, 18 Nov 2021 at 11:16, Mark Jens <[email protected]> wrote: > Hi, > > On Thu, 18 Nov 2021 at 10:38, Istvan Toth <[email protected]> > wrote: > >> Phoenix probably won't work with HBase 3.x, but looking at the linked >> commit, it should be fairly straightforward to apply that to HBase 2.4. >> >> I'm not sure why that hasn't been backported to 2.x, perhaps HBase doesn't >> have test infra set up for ARM. >> You may want to discuss backporting that change to the active Hbase 2.x >> branches with the HBase project. >> > > I've asked for a backport at > https://issues.apache.org/jira/browse/HBASE-23612?focusedCommentId=17445761&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17445761 > > Apologies for creating https://issues.apache.org/jira/browse/PHOENIX-6595. > Initially I thought that Phoenix can apply the patch since it already > requires manual build of HBase. > But you are right - it would be much better if the improvement is > backported to HBase 2.x ! > > >> >> >> On Wed, Nov 17, 2021 at 12:35 PM Mark Jens <[email protected]> wrote: >> >> > Hi Istvan, >> > >> > It seems HBase support for ARM64 is available only in 3.x: >> > >> > >> https://github.com/apache/hbase/commit/5480493f5f7b01b496f54215334543f2a82c6ba7 >> > Would Phoenix work with HBase 3.x ? >> > >> > On Wed, 17 Nov 2021 at 13:28, Mark Jens <[email protected]> wrote: >> > >> > > Thanks for the hint! >> > > >> > > Unfortunately HBase-2.4.8 build fails with: >> > > >> > > INFO] BUILD FAILURE >> > > [INFO] >> > > >> ------------------------------------------------------------------------ >> > > [INFO] Total time: 01:42 min >> > > [INFO] Finished at: 2021-11-17T11:23:19Z >> > > [INFO] >> > > >> ------------------------------------------------------------------------ >> > > [ERROR] Failed to execute goal >> > > org.xolstice.maven.plugins:protobuf-maven-plugin:0.6.1:compile >> > > (compile-protoc) on project hbase-protocol: Unable to resolve >> artifact: >> > > Missing: >> > > [ERROR] ---------- >> > > [ERROR] 1) com.google.protobuf:protoc:exe:linux-aarch_64:2.5.0 >> > > [ERROR] >> > > [ERROR] Try downloading the file manually from the project website. >> > > [ERROR] >> > > [ERROR] Then, install it using the command: >> > > [ERROR] mvn install:install-file -DgroupId=com.google.protobuf >> > > -DartifactId=protoc -Dversion=2.5.0 -Dclassifier=linux-aarch_64 >> > > -Dpackaging=exe -Dfile=/path/to/file >> > > [ERROR] >> > > [ERROR] Alternatively, if you host your own repository you can >> deploy >> > > the file there: >> > > [ERROR] mvn deploy:deploy-file -DgroupId=com.google.protobuf >> > > -DartifactId=protoc -Dversion=2.5.0 -Dclassifier=linux-aarch_64 >> > > -Dpackaging=exe -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id] >> > > [ERROR] >> > > [ERROR] Path to dependency: >> > > [ERROR] 1) org.apache.hbase:hbase-protocol:jar:2.4.8 >> > > [ERROR] 2) com.google.protobuf:protoc:exe:linux-aarch_64:2.5.0 >> > > >> > > >> > > >> > > >> > > On Wed, 17 Nov 2021 at 12:27, Istvan Toth <[email protected] >> > >> > > wrote: >> > > >> > >> You need to recompile HBase. >> > >> See BULIDING.md >> > >> >> > >> On Wed, Nov 17, 2021 at 10:33 AM Mark Jens <[email protected]> >> > wrote: >> > >> >> > >> > Hello, >> > >> > >> > >> > I am trying to build Phoenix on Ubuntu 20.04.3 ARM64. >> > >> > >> > >> > Phoenix Core module fails with: >> > >> > >> > >> > [ERROR] >> > >> > >> > >> > >> > >> >> > >> org.apache.phoenix.hbase.index.write.recovery.TestPerRegionIndexWriteCache.testMultipleAddsForSingleRegion >> > >> > Time elapsed: 0.025 s <<< ERROR! >> > >> > java.lang.IncompatibleClassChangeError: Found interface >> > >> > org.apache.hadoop.hdfs.protocol.HdfsFileStatus, but class was >> expected >> > >> > at >> > >> > org.apache.hadoop.hbase.io >> > >> > >> > >> >> > >> .asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:536) >> > >> > at >> > >> > org.apache.hadoop.hbase.io >> > >> > >> > >> >> > >> .asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$400(FanOutOneBlockAsyncDFSOutputHelper.java:112) >> > >> > at >> > >> > org.apache.hadoop.hbase.io >> > >> > >> > >> >> > >> .asyncfs.FanOutOneBlockAsyncDFSOutputHelper$8.doCall(FanOutOneBlockAsyncDFSOutputHelper.java:616) >> > >> > at >> > >> > org.apache.hadoop.hbase.io >> > >> > >> > >> >> > >> .asyncfs.FanOutOneBlockAsyncDFSOutputHelper$8.doCall(FanOutOneBlockAsyncDFSOutputHelper.java:611) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) >> > >> > at >> > >> > org.apache.hadoop.hbase.io >> > >> > >> > >> >> > >> .asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:624) >> > >> > at >> > >> > org.apache.hadoop.hbase.io >> > >> > >> .asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:53) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:180) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:166) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:113) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:669) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:130) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:841) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:548) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:489) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:160) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62) >> > >> > at >> org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:296) >> > >> > at >> > >> > >> > >> > >> > >> >> > >> org.apache.phoenix.hbase.index.write.recovery.TestPerRegionIndexWriteCache.setUp(TestPerRegionIndexWriteCache.java:109) >> > >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> > >> > .... >> > >> > [INFO] >> > >> > [INFO] Results: >> > >> > [INFO] >> > >> > [ERROR] Errors: >> > >> > [ERROR] TestPerRegionIndexWriteCache.setUp:109 » >> > >> IncompatibleClassChange >> > >> > Found interfa... >> > >> > [ERROR] TestPerRegionIndexWriteCache.setUp:109 » >> > >> IncompatibleClassChange >> > >> > Found interfa... >> > >> > [ERROR] TestPerRegionIndexWriteCache.setUp:109 » >> > >> IncompatibleClassChange >> > >> > Found interfa... >> > >> > [INFO] >> > >> > [ERROR] Tests run: 1909, Failures: 0, Errors: 3, Skipped: 6 >> > >> > [INFO] >> > >> > [INFO] >> > >> > >> > ------------------------------------------------------------------------ >> > >> > [INFO] Reactor Summary for Apache Phoenix 5.2.0-SNAPSHOT: >> > >> > [INFO] >> > >> > [INFO] Apache Phoenix ..................................... >> SUCCESS [ >> > >> > 2.034 s] >> > >> > [INFO] Phoenix Hbase 2.4.1 compatibility .................. >> SUCCESS [ >> > >> > 5.358 s] >> > >> > [INFO] Phoenix Hbase 2.4.0 compatibility .................. >> SUCCESS [ >> > >> > 3.946 s] >> > >> > [INFO] Phoenix Hbase 2.3.0 compatibility .................. >> SUCCESS [ >> > >> > 4.437 s] >> > >> > [INFO] Phoenix Hbase 2.2.5 compatibility .................. >> SUCCESS [ >> > >> > 4.004 s] >> > >> > [INFO] Phoenix Hbase 2.1.6 compatibility .................. >> SUCCESS [ >> > >> > 3.966 s] >> > >> > [INFO] Phoenix Core ....................................... FAILURE >> > >> [01:25 >> > >> > min] >> > >> > [INFO] Phoenix - Pherf .................................... SKIPPED >> > >> > ... >> > >> > >> > >> > Any idea why this breaks ? >> > >> > >> > >> > It does not look ARM64 specific to me. I will try on x64 too. >> > >> > >> > >> > Thanks! >> > >> > >> > >> > Mark >> > >> > >> > >> >> > >> >> > >> -- >> > >> *István Tóth* | Staff Software Engineer >> > >> [email protected] <https://www.cloudera.com> >> > >> [image: Cloudera] <https://www.cloudera.com/> >> > >> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: >> > >> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: >> > >> Cloudera >> > >> on LinkedIn] <https://www.linkedin.com/company/cloudera> >> > >> <https://www.cloudera.com/> >> > >> ------------------------------ >> > >> >> > > >> > >> >> >> -- >> *István Tóth* | Staff Software Engineer >> [email protected] <https://www.cloudera.com> >> [image: Cloudera] <https://www.cloudera.com/> >> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: >> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: >> Cloudera >> on LinkedIn] <https://www.linkedin.com/company/cloudera> >> <https://www.cloudera.com/> >> ------------------------------ >> >
