In general, it is fine to setup a hbase cluster which ships with hadoop 2.x client library against a hadoop 3.x hdfs and yarn cluster. The wire communication is still compatible. That's why we still keep the client library as 2.x and not do something like the time in 0.98 that publish two binaries for hadoop1 and hadoop2.
So I do not understand why 'no end-users with Hadoop 3 clusters will be able to use the Apache-distributed binaries'. And on the Phoenix IT tests, it is a problem as HBase does not support drop in replacement with hadoop libraries. There are several possible directions: 1. Support drop in replacement. 2. Publish two binaries with hadoop2 and hadoop3, and also publish two maven dependencies with hadoop2 and hadoop3. 3. Only publish hadoop3 binaries and maven dependencies. The problem for these directions 1. Not sure if this is easy to implement... 2. Will increase the complexity when users just want to use hbase. 3. May have compatible issues when using the hadoop3 libraries against a hadoop 2 cluster. Thanks. Geoffrey Jacoby <gjac...@apache.org> 于2022年8月17日周三 03:30写道： > > I see that the next HBase 2.5 RC is imminent, and before that's set in > stone, I wanted to bring up the question of whether there will be official > HBase 2.5 binaries built with the Hadoop 3 profile and available in the > usual Maven repositories. (In addition to the usual Hadoop 2 profile > binaries) > > The HBase 2.x line has a commitment to maintain support for Hadoop 2.x, but > Hadoop 3.3 is the current stable Hadoop line and the most recent release > notes [1] encourage all users of Hadoop 2.x to upgrade to Hadoop 3. > > Without convenience artifacts built against Hadoop 3, no end-users with > Hadoop 3 clusters will be able to use the Apache-distributed binaries and > will instead have to recompile HBase from source themselves, or use a 3rd > party distribution that does so for them. > > This is especially inconvenient for downstream projects such as Apache > Phoenix, which has never officially supported the HBase 2.x / Hadoop 2.10 > combination. (It currently supports only HBase 2.3 or 2.4 with Hadoop 3. > HBase 2.5 support will be added very shortly after its release as part of > Phoenix 5.2.) > > To even run the Phoenix IT tests locally requires contributors to download > the HBase source release and manually mvn install to their local maven repo > using the Hadoop 3 profile, to avoid crashes in the HBase minicluster.[2] > This is a barrier to new contributors and confuses even veteran ones, and > has to be done again for every new HBase release. > > In general, I expect the Hadoop 3 user base to grow and the Hadoop 2.10 > user base to shrink with every future HBase 2 release, so I think this is a > worthwhile improvement. > > Thanks, > > Geoffrey > > [1] https://hadoop.apache.org/release/3.3.4.html > [2] https://github.com/apache/phoenix/blob/master/BUILDING.md
