> This would include - again, in my personal opinion - not dropping support > for HBase releases in Phoenix patch releases as a policy.
Now that I gave some more thoughts to this, I agree to having such a policy to build more confidence within Phoenix as well as HBase users. Although we think of dropping support of HBase 2.4.1 but the main point is we have no control over IA.Private APIs of HBase that we use at present because they can be changed anytime. So it's not just about 2.4.1 but this might happen anytime in future. I think we should support more HBase patch releases to build better confidence within our community. What if some users do want to run HBase 2.4.1 only (for some/any reasons) but find out that Phoenix doesn't support (and publish binaries for) only this patch version of HBase on the 2.4 release line? Also, as I proposed initially on this thread, we don't have to consume Jenkins resources for each profile, we can just run Jenkins tests for latest profiles on each HBase minor version that we support. On Wed, Feb 17, 2021 at 3:14 PM Istvan Toth <st...@cloudera.com.invalid> wrote: > Thanks for your thoughtful response, Geoffrey. > > I agree with all of your points for the most part. > > Going a bit off tangent, and referencing to the EOL HBase support issue > thread, I personally think that if we want to increase > Phoenix adoption, we should build confidence in the Hbase community that > they can count on having a Phoenix that works on their HBase cluster > for a reasonable amount of time, preferably with updates. > > This would include - again, in my personal opinion - not dropping support > for HBase releases in Phoenix patch releases as a policy. > > Back in the other thread, Ankit argues that this would cause undue extra > work. IMO having more users and eventually contributors > would be well worth the extra effort that sticking to some predictable > policies would need. > > In this case, we are fully equipped to handle the multiple 2.4.x versions, > Viraj's current patch needs just a few character changes, > and we can support both 2.4.0 and 2.4.1 in 5.1.1 > > Publishing binaries only the latest profiles, or for all profiles is a > matter of enumerating them in maven property. > > Istvan > > > On Wed, Feb 17, 2021 at 7:34 AM Geoffrey Jacoby <gjac...@apache.org> > wrote: > > > Istvan, > > > > What makes the case tricky is that the broken class is IA.Private. HBase > > has set out compatibility guidelines about what is and isn't safe for > > external projects to depend upon, and in this particular case it looks > like > > we were trying to do something using an IA.Private API that isn't safe to > > use externally. HBase is allowed to break Phoenix on a patch level when > > Phoenix depends on private APIs. > > > > One thing to follow-up on would be to work with the HBase community to > > allow us to replace our IA.Private calls with a new IA.LimitedPrivate or > > IA.Public API that does have compatibility guarantees. From looking at > the > > Phoenix code, it looks like we're just trying to create StoreFiles as the > > output of a MapReduce job, which seems a reasonable thing for there to be > > HBase support for, in addition to HBase's own internal MapReduce formats. > > However, that new public or limited private API would only be usable with > > HBase 2.5 and up (since interface changes need a minor release.) > > > > I agree with either Andrew's kind offer to bring back temporary support > of > > the old HStore methods in 2.4.2, or alternatively to only support 2.4.1 > and > > up in Phoenix 5.1.1 (since 5.1 exists with 2.4.0 support and there's a > > workaround for PHOENIX-6377 for anyone who needs it). Would be good to > > avoid the extra compatibility profile this time, even if we eventually > need > > more in the future for other reasons. > > > > Geoffrey > > > > On Tue, Feb 16, 2021 at 9:53 PM Istvan Toth <st...@cloudera.com.invalid> > > wrote: > > > > > Putting back the methods for 2.4.2 and blacklisting 2.4.1 could be a > > > solution for this specific case. (thanks for the offer) > > > > > > However similar past changes were more involved, where this wouldn't be > > an > > > option. The last two that I remember: > > > - multi-join support in 2.0, 2.1, 2.2 > > > - raw filter support in 2.2 > > > > > > I think that we should expect that these api breakages happen, and > handle > > > these cases as the norm, > > > and have a policy for them. > > > > > > (Not that we shouldn't ask HBase not to break us in a patch level, if > it > > is > > > just for the sake of refactoring) > > > > > > Istvan > > > > > > On Wed, Feb 17, 2021 at 5:49 AM Andrew Purtell < > andrew.purt...@gmail.com > > > > > > wrote: > > > > > > > If you opened a PR that puts the HStore methods back, marks them > > > > @Deprecated, and implements them by calling the new StoreUtil > methods, > > I > > > > would approve it. Just for branch-2.4. Call it a courtesy for > Phoenix. > > > > > > > > Assuming this gets released in 2.4.2, around the end of the month, > then > > > > you could opt to blacklist 2.4.1 in your build rather than deal with > > it. > > > > Then this becomes an issue for a future 2.5 compat module. > > > > > > > > > > > > > On Feb 16, 2021, at 8:38 PM, Viraj Jasani <vjas...@apache.org> > > wrote: > > > > > > > > > > Thanks Andrew! That’s it, HBASE-25249 is the only refactoring that > > > broke > > > > > compat and since HStore is IA.Private, I could not argue from HBase > > > side > > > > to > > > > > bring methods back in HStore. If we can do it in 2.4.2, that would > be > > > > great > > > > > and we can skip 2.4.1 support i think. > > > > > WDYT Istvan? > > > > > > > > > > > > > > >> On Wed, 17 Feb 2021 at 3:28 AM, Andrew Purtell < > apurt...@apache.org > > > > > > > wrote: > > > > >> > > > > >> Hmm... I see. This thread is probably about PHOENIX-6359, so the > > > change > > > > was > > > > >> HBASE-25249. > > > > >> > > > > >> The methods moved from HStore to StoreUtils can be added back to > > > HStore > > > > as > > > > >> compatibility methods in branch-2.4. > > > > >> > > > > >> Is there more? > > > > >> > > > > >> > > > > >> On Tue, Feb 16, 2021 at 1:34 PM Andrew Purtell < > apurt...@apache.org > > > > > > > >> wrote: > > > > >> > > > > >>>> While supporting a new HBase patch release version (e.g 2.4.1), > if > > > it > > > > >>> turns out to be incompatible with existing HBase minor release > > > profile > > > > >> (e.g > > > > >>> profile 2.4), we might have to consider some extra steps. > > > > >>> > > > > >>> What happened? If we broke something in a public or limited > private > > > > >>> interface between 2.4.0 and 2.4.1, we can fix it, and you'll be > > good > > > > >> again > > > > >>> in 2.4.2 and forward. You could blacklist 2.4.1 in your build of > > the > > > > 2.4 > > > > >>> compat module using the enforcer plugin if you like. > > > > >>> > > > > >>> If the break was a private interface, but it is a simple issue, > > like > > > > >>> removal of a constant field, or removal of a method that's easy > to > > > put > > > > >> back > > > > >>> for sake of compatibility, we can probably just put it back. By > > > > >> 'probably' > > > > >>> I mean I would not be opposed to it but there's always the chance > > > that > > > > >>> someone would object. > > > > >>> > > > > >>> > > > > >>>> On Tue, Feb 16, 2021 at 4:07 AM Viraj Jasani < > vjas...@apache.org> > > > > wrote: > > > > >>> > > > > >>>> Hi, > > > > >>>> > > > > >>>> While supporting a new HBase patch release version (e.g 2.4.1), > if > > > it > > > > >>>> turns > > > > >>>> out to be incompatible with existing HBase minor release profile > > > (e.g > > > > >>>> profile 2.4), we might have to consider some extra steps. > > > > >>>> > > > > >>>> Proposals: > > > > >>>> 1. Add a new profile for each compat module > > > > >>>> 2. Profile with HBase minor version always support latest > > supported > > > > >> HBase > > > > >>>> compat module and HBase patch release (e.g in our case, profile > > 2.4 > > > > uses > > > > >>>> compat-module 2.4.1, and profile 2.4.0 uses compat module 2.4.0) > > > > >>>> 3. We run jenkins tests only for latest minor release profiles > > (e.g > > > > with > > > > >>>> profiles 2.4 and 2.4.0 in place, we run tests for profile 2.4 > > only, > > > > >> which > > > > >>>> points to latest version 2.4.1) > > > > >>>> > > > > >>>> HBase profile to build compatibility examples: > > > > >>>> > > > > >>>> *Profile for HBase minor version:* > > > > >>>> 2.1 (supports all 2.1.x releases), > > > > >>>> 2.3 (supports all 2.3.x releases), > > > > >>>> 2.4 (if we have profile 2.4.0, 2.4 profile supports 2.4.1+ > > releases > > > > and > > > > >>>> not > > > > >>>> 2.4.0) > > > > >>>> > > > > >>>> *Profile for HBase patch version:* > > > > >>>> 2.4.0 (supports only 2.4.0 release) > > > > >>>> > > > > >>>> Thoughts? > > > > >>>> > > > > >>> > > > > >>> > > > > >>> -- > > > > >>> Best regards, > > > > >>> Andrew > > > > >>> > > > > >>> Words like orphans lost among the crosstalk, meaning torn from > > > truth's > > > > >>> decrepit hands > > > > >>> - A23, Crosstalk > > > > >>> > > > > >> > > > > >> > > > > >> -- > > > > >> Best regards, > > > > >> Andrew > > > > >> > > > > >> Words like orphans lost among the crosstalk, meaning torn from > > truth's > > > > >> decrepit hands > > > > >> - A23, Crosstalk > > > > >> > > > > > > > > > > > > > -- > > > *István Tóth* | Staff Software Engineer > > > st...@cloudera.com <https://www.cloudera.com> > > > [image: Cloudera] <https://www.cloudera.com/> > > > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: > > > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: > > Cloudera > > > on LinkedIn] <https://www.linkedin.com/company/cloudera> > > > <https://www.cloudera.com/> > > > ------------------------------ > > > > > > > > -- > *István Tóth* | Staff Software Engineer > st...@cloudera.com <https://www.cloudera.com> > [image: Cloudera] <https://www.cloudera.com/> > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera > on LinkedIn] <https://www.linkedin.com/company/cloudera> > <https://www.cloudera.com/> > ------------------------------ >
