I think 2.x is going to have a few years of life remaining so it would be best, if we are going to address this, to have a 2.x solution was well as a 3.x one.
In my opinion we can continue to publish 2.4 and 2.5 (and 2.6) unchanged and then also introduce a Hadoop 3 release using “hadoop3” or similar as Maven classifier. Phoenix could specify this classifier in their POMs. Everyone should be happy. Users who already are comfortable with the Hadoop 2 default don’t have to change anything. A one time POM change on the Phoenix side is required but that’s it. The additional build time complexity for generating two releases can be incorporated into create-release. Nobody does manual releases any more as far as I know. Likewise, download and verification of -hadoop3 convenience binaries can be added to hbase-vote. I believe we are all using that tool for verification of releases now. After these one time changes are landed the cost for RMs and PMC will be only in a roughly doubled amount of time needed to build and verify releases. > On Aug 17, 2022, at 9:06 AM, Nick Dimiduk <ndimi...@apache.org> wrote: > > Hi Geoffrey, > > I have no complaints with shipping convenience binaries built against both > Hadoop2 and Hadoop3. The primary challenge is implementing the > necessary build changes, the secondary challenge is verifying/testing it > works reliably. > > But for Phoenix, are you asking for convenience binaries, or are you asking > for artifacts published into maven that have the Hadoop3 profile activated > and specify the associated dependencies? > > I'm afraid that the 2.5.0 release ship has already sailed. I've heard talk > of a 2.6 "fast-follow", so maybe someone can have the build changes ready > for that? Also, isn't this a too little, too late situation? Shouldn't we > shift our focus to releasing 3.0, which has dropped support for Hadoop2? > > Thanks, > Nick > >> On Tue, Aug 16, 2022 at 9:30 PM Geoffrey Jacoby <gjac...@apache.org> wrote: >> >> I see that the next HBase 2.5 RC is imminent, and before that's set in >> stone, I wanted to bring up the question of whether there will be official >> HBase 2.5 binaries built with the Hadoop 3 profile and available in the >> usual Maven repositories. (In addition to the usual Hadoop 2 profile >> binaries) >> >> The HBase 2.x line has a commitment to maintain support for Hadoop 2.x, but >> Hadoop 3.3 is the current stable Hadoop line and the most recent release >> notes [1] encourage all users of Hadoop 2.x to upgrade to Hadoop 3. >> >> Without convenience artifacts built against Hadoop 3, no end-users with >> Hadoop 3 clusters will be able to use the Apache-distributed binaries and >> will instead have to recompile HBase from source themselves, or use a 3rd >> party distribution that does so for them. >> >> This is especially inconvenient for downstream projects such as Apache >> Phoenix, which has never officially supported the HBase 2.x / Hadoop 2.10 >> combination. (It currently supports only HBase 2.3 or 2.4 with Hadoop 3. >> HBase 2.5 support will be added very shortly after its release as part of >> Phoenix 5.2.) >> >> To even run the Phoenix IT tests locally requires contributors to download >> the HBase source release and manually mvn install to their local maven repo >> using the Hadoop 3 profile, to avoid crashes in the HBase minicluster.[2] >> This is a barrier to new contributors and confuses even veteran ones, and >> has to be done again for every new HBase release. >> >> In general, I expect the Hadoop 3 user base to grow and the Hadoop 2.10 >> user base to shrink with every future HBase 2 release, so I think this is a >> worthwhile improvement. >> >> Thanks, >> >> Geoffrey >> >> [1] https://hadoop.apache.org/release/3.3.4.html >> [2] https://github.com/apache/phoenix/blob/master/BUILDING.md >>
