Thinking about this a bit more, we will have an issue in that the POMs published from our -hadoop3 build will not have a default activation of our Hadoop 3 build profile. The convenience binaries will function as expected but Maven will read and process eg Phoenix POMs, then download and perform substitutions on HBase POMs, and then etc, so downstreamers like Phoenix will have to set up the hadoop.profile variable for us in their default build profile or else the transitive paths through us may be wrong. I wonder if there is a Maven plugin available for deploying POMs with all variable substitutions performed before deployment, that would solve that problem and all conceivable related issues.
> On Aug 25, 2022, at 11:03 AM, Andrew Purtell <andrew.purt...@gmail.com> wrote: > > I think 2.x is going to have a few years of life remaining so it would be > best, if we are going to address this, to have a 2.x solution was well as a > 3.x one. > > In my opinion we can continue to publish 2.4 and 2.5 (and 2.6) unchanged and > then also introduce a Hadoop 3 release using “hadoop3” or similar as Maven > classifier. Phoenix could specify this classifier in their POMs. Everyone > should be happy. Users who already are comfortable with the Hadoop 2 default > don’t have to change anything. A one time POM change on the Phoenix side is > required but that’s it. > > The additional build time complexity for generating two releases can be > incorporated into create-release. Nobody does manual releases any more as far > as I know. Likewise, download and verification of -hadoop3 convenience > binaries can be added to hbase-vote. I believe we are all using that tool for > verification of releases now. After these one time changes are landed the > cost for RMs and PMC will be only in a roughly doubled amount of time needed > to build and verify releases. > >> On Aug 17, 2022, at 9:06 AM, Nick Dimiduk <ndimi...@apache.org> wrote: >> >> Hi Geoffrey, >> >> I have no complaints with shipping convenience binaries built against both >> Hadoop2 and Hadoop3. The primary challenge is implementing the >> necessary build changes, the secondary challenge is verifying/testing it >> works reliably. >> >> But for Phoenix, are you asking for convenience binaries, or are you asking >> for artifacts published into maven that have the Hadoop3 profile activated >> and specify the associated dependencies? >> >> I'm afraid that the 2.5.0 release ship has already sailed. I've heard talk >> of a 2.6 "fast-follow", so maybe someone can have the build changes ready >> for that? Also, isn't this a too little, too late situation? Shouldn't we >> shift our focus to releasing 3.0, which has dropped support for Hadoop2? >> >> Thanks, >> Nick >> >>>> On Tue, Aug 16, 2022 at 9:30 PM Geoffrey Jacoby <gjac...@apache.org> wrote: >>> >>> I see that the next HBase 2.5 RC is imminent, and before that's set in >>> stone, I wanted to bring up the question of whether there will be official >>> HBase 2.5 binaries built with the Hadoop 3 profile and available in the >>> usual Maven repositories. (In addition to the usual Hadoop 2 profile >>> binaries) >>> >>> The HBase 2.x line has a commitment to maintain support for Hadoop 2.x, but >>> Hadoop 3.3 is the current stable Hadoop line and the most recent release >>> notes [1] encourage all users of Hadoop 2.x to upgrade to Hadoop 3. >>> >>> Without convenience artifacts built against Hadoop 3, no end-users with >>> Hadoop 3 clusters will be able to use the Apache-distributed binaries and >>> will instead have to recompile HBase from source themselves, or use a 3rd >>> party distribution that does so for them. >>> >>> This is especially inconvenient for downstream projects such as Apache >>> Phoenix, which has never officially supported the HBase 2.x / Hadoop 2.10 >>> combination. (It currently supports only HBase 2.3 or 2.4 with Hadoop 3. >>> HBase 2.5 support will be added very shortly after its release as part of >>> Phoenix 5.2.) >>> >>> To even run the Phoenix IT tests locally requires contributors to download >>> the HBase source release and manually mvn install to their local maven repo >>> using the Hadoop 3 profile, to avoid crashes in the HBase minicluster.[2] >>> This is a barrier to new contributors and confuses even veteran ones, and >>> has to be done again for every new HBase release. >>> >>> In general, I expect the Hadoop 3 user base to grow and the Hadoop 2.10 >>> user base to shrink with every future HBase 2 release, so I think this is a >>> worthwhile improvement. >>> >>> Thanks, >>> >>> Geoffrey >>> >>> [1] https://hadoop.apache.org/release/3.3.4.html >>> [2] https://github.com/apache/phoenix/blob/master/BUILDING.md >>>
