yes, the flatten plugin. We use it in hbase-connectors already. https://www.mojohaus.org/flatten-maven-plugin/
this sounds like it could also be a use case for BOMs, which would also benefit users of our client artifacts that use build tools that don't respect maven profiles generally, like gradle. On Thu, Aug 25, 2022 at 10:30 AM Andrew Purtell <andrew.purt...@gmail.com> wrote: > Thinking about this a bit more, we will have an issue in that the POMs > published from our -hadoop3 build will not have a default activation of our > Hadoop 3 build profile. The convenience binaries will function as expected > but Maven will read and process eg Phoenix POMs, then download and perform > substitutions on HBase POMs, and then etc, so downstreamers like Phoenix > will have to set up the hadoop.profile variable for us in their default > build profile or else the transitive paths through us may be wrong. I > wonder if there is a Maven plugin available for deploying POMs with all > variable substitutions performed before deployment, that would solve that > problem and all conceivable related issues. > > > On Aug 25, 2022, at 11:03 AM, Andrew Purtell <andrew.purt...@gmail.com> > wrote: > > > > I think 2.x is going to have a few years of life remaining so it would > be best, if we are going to address this, to have a 2.x solution was well > as a 3.x one. > > > > In my opinion we can continue to publish 2.4 and 2.5 (and 2.6) unchanged > and then also introduce a Hadoop 3 release using “hadoop3” or similar as > Maven classifier. Phoenix could specify this classifier in their POMs. > Everyone should be happy. Users who already are comfortable with the Hadoop > 2 default don’t have to change anything. A one time POM change on the > Phoenix side is required but that’s it. > > > > The additional build time complexity for generating two releases can be > incorporated into create-release. Nobody does manual releases any more as > far as I know. Likewise, download and verification of -hadoop3 convenience > binaries can be added to hbase-vote. I believe we are all using that tool > for verification of releases now. After these one time changes are landed > the cost for RMs and PMC will be only in a roughly doubled amount of time > needed to build and verify releases. > > > >> On Aug 17, 2022, at 9:06 AM, Nick Dimiduk <ndimi...@apache.org> wrote: > >> > >> Hi Geoffrey, > >> > >> I have no complaints with shipping convenience binaries built against > both > >> Hadoop2 and Hadoop3. The primary challenge is implementing the > >> necessary build changes, the secondary challenge is verifying/testing it > >> works reliably. > >> > >> But for Phoenix, are you asking for convenience binaries, or are you > asking > >> for artifacts published into maven that have the Hadoop3 profile > activated > >> and specify the associated dependencies? > >> > >> I'm afraid that the 2.5.0 release ship has already sailed. I've heard > talk > >> of a 2.6 "fast-follow", so maybe someone can have the build changes > ready > >> for that? Also, isn't this a too little, too late situation? Shouldn't > we > >> shift our focus to releasing 3.0, which has dropped support for Hadoop2? > >> > >> Thanks, > >> Nick > >> > >>>> On Tue, Aug 16, 2022 at 9:30 PM Geoffrey Jacoby <gjac...@apache.org> > wrote: > >>> > >>> I see that the next HBase 2.5 RC is imminent, and before that's set in > >>> stone, I wanted to bring up the question of whether there will be > official > >>> HBase 2.5 binaries built with the Hadoop 3 profile and available in the > >>> usual Maven repositories. (In addition to the usual Hadoop 2 profile > >>> binaries) > >>> > >>> The HBase 2.x line has a commitment to maintain support for Hadoop > 2.x, but > >>> Hadoop 3.3 is the current stable Hadoop line and the most recent > release > >>> notes [1] encourage all users of Hadoop 2.x to upgrade to Hadoop 3. > >>> > >>> Without convenience artifacts built against Hadoop 3, no end-users with > >>> Hadoop 3 clusters will be able to use the Apache-distributed binaries > and > >>> will instead have to recompile HBase from source themselves, or use a > 3rd > >>> party distribution that does so for them. > >>> > >>> This is especially inconvenient for downstream projects such as Apache > >>> Phoenix, which has never officially supported the HBase 2.x / Hadoop > 2.10 > >>> combination. (It currently supports only HBase 2.3 or 2.4 with Hadoop > 3. > >>> HBase 2.5 support will be added very shortly after its release as part > of > >>> Phoenix 5.2.) > >>> > >>> To even run the Phoenix IT tests locally requires contributors to > download > >>> the HBase source release and manually mvn install to their local maven > repo > >>> using the Hadoop 3 profile, to avoid crashes in the HBase > minicluster.[2] > >>> This is a barrier to new contributors and confuses even veteran ones, > and > >>> has to be done again for every new HBase release. > >>> > >>> In general, I expect the Hadoop 3 user base to grow and the Hadoop 2.10 > >>> user base to shrink with every future HBase 2 release, so I think this > is a > >>> worthwhile improvement. > >>> > >>> Thanks, > >>> > >>> Geoffrey > >>> > >>> [1] https://hadoop.apache.org/release/3.3.4.html > >>> [2] https://github.com/apache/phoenix/blob/master/BUILDING.md > >>> >
