Hi! (Sorry for not replying earlier, I'll need to revisit my mail filters)
I do have a half-baked patch with the lightweight isolation in modules case. In its current form it's more ugly than complex, but some ugliness is a small price to pay for not having to maintain multiple branches. (And it can be beutified later) I will complete it, and make the PR for that as time permits. The tephra solution handles the version differences during run (startup) time, my current solution is compile-time. Istvan On Fri, Dec 20, 2019 at 11:05 AM la...@apache.org <la...@apache.org> wrote: > Yep that. > For the record... I do not think this is simple. But it is possible. > > On Thursday, December 19, 2019, 8:37:37 PM GMT+1, Andrew Purtell < > apurt...@apache.org> wrote: > > I can't answer for Lars but whenever version incompatibilities come up > usually only a handful of files are impacted. In the last round, the > Phoenix access controller, a related file in the same directory, and the > RPC scheduler. If you cloned these into separate version specific maven > modules case by case as needed at each round the differences are fairly > small. On the other hand if you take a principled approach and abstract all > the things, it will be a huge effort and nobody realistically will want to > take it on. > > On Thu, Dec 19, 2019 at 11:34 AM Geoffrey Jacoby <gjac...@gmail.com> > wrote: > > > Lars, > > > > I'm curious why you say the differences are easily isolated -- many of > the > > core classes of Phoenix either directly inherit HBase classes or > implement > > HBase interfaces, and those can vary between minor versions. (See my > above > > example of a new coprocessor hook on BaseRegionObserver.) > > > > Geoffrey > > > > On Thu, Dec 19, 2019 at 10:54 AM la...@apache.org <la...@apache.org> > > wrote: > > > > > Yep. The differences are pretty minimal - provided they can be > isolated > > > easily. > > > Tephra might be a pretty good model. It supports various versions of > > HBase > > > in a single branch and has similar issues as Phoenix (coprocessors, > etc). > > > -- Lars > > > On Thursday, December 19, 2019, 7:07:51 PM GMT+1, Josh Elser < > > > els...@apache.org> wrote: > > > > > > To clarify, you think that compat modules are better than that > > > separate-branches model in 4.x? > > > > > > On 12/18/19 11:29 AM, la...@apache.org wrote: > > > > This is really hard to follow. > > > > > > > > I think we should do the same with HBase dependencies in Phoenix that > > > HBase does with Hadoop dependencies. > > > > > > > > That is: We could have a maven module with the specific HBase > version > > > dependent code. > > > > Btw. Tephra does the same... A module for HBase version specific > code. > > > > -- Lars > > > > > > > > On Tuesday, December 17, 2019, 10:00:31 AM GMT+1, Istvan Toth < > > > st...@apache.org> wrote: > > > > > > > > What do you think about tying the minor releases to Hbase minor > > releases > > > > (not necessarily one-to-one) > > > > > > > > for example (provided 5.1 is 2020H1) > > > > > > > > 5.0.0 -> HB 2.0 > > > > 5.1.0 -> HB 2.2.2 (and whatever 2.1 is API compatible with it) > > > > 5.1.x -> HB 2.2.x (treat as maintenance branch, no major new > features) > > > > 5.2.0 -> HB 2.3.0 (if released by that time) > > > > 5.2.x -> HB 2.3.x (treat as maintenance branch, no major new > features) > > > > 5.3.0 -> HB 2.3.x (if there is no new major/minor Hbase release) > > > > master -> latest released HBase version > > > > > > > > Alternatively, we could stick with the same HBase version for patch > > > > releases that we used for the first minor release. > > > > > > > > This would limit the number of branches that we have to maintain in > > > > parallel, while providing maintenance branches for older releases, > and > > > > timely-ish Phoenix releases. > > > > > > > > The drawback is that users of old HBase versions won't get the latest > > > > features, on the other hand they can expect more polish. > > > > > > > > Istvan > > > > > > > > On Thu, Dec 12, 2019 at 8:05 PM Geoffrey Jacoby <gjac...@apache.org> > > > wrote: > > > > > > > >> Since HBase 2.0 is EOM'ed, I'm +1 for not worrying about 2.0.x > > > >> compatibility with the 5.x branch going forward. > > > >> > > > >> Given how coupled Phoenix is to the implementation details of HBase > > > though, > > > >> I'm not sure trying to abstract those away to keep one Phoenix > branch > > > per > > > >> HBase major version is practical, however. At the least, it would be > > > really > > > >> complex. > > > >> > > > >> For example, in the new year I plan to return to working on the > change > > > data > > > >> capture and Phoenix-level replication features, both of which depend > > on > > > >> WALKey interface changes and a new RegionObserver coprocessor hook > > > >> introduced in HBASE-22622 and HBASE-22623. This was released in > HBase > > > 1.5 > > > >> and will be in the forthcoming HBase 2.3. While the HBase community > is > > > >> discussing EOMing 1.3 right now, and maybe 1.4 will go in the medium > > > term, > > > >> I don't see all pre-2.3 branch-2's getting deprecated anytime soon. > > > >> > > > >> So there will be at least two significant features that can only > exist > > > in > > > >> some but not all of our 4.x and 5.x branches. > > > >> > > > >> Geoffrey > > > >> > > > >> On Thu, Dec 12, 2019 at 8:21 AM Josh Elser <els...@apache.org> > wrote: > > > >> > > > >>> As much as possible, I'd like to avoid us getting into another > > > situation > > > >>> with 5.x where we have multiple branches. My hope was/is that we > can > > > >>> keep one Phoenix5 branch that works against an acceptable set of > > HBase > > > >>> branches. > > > >>> > > > >>> To me, that acceptable set of HBase branches is _a_ 2.1 and 2.2 > > > release. > > > >>> I don't think we need to support all 2.1.x or 2.2.x, nor do I think > > we > > > >>> need to keep trying to maintain 2.0.x as it's already end of > support > > by > > > >>> the HBase community. > > > >>> > > > >>> Thanks for updating your PR. I'll add this to my review queue. > > > >>> > > > >>> On 12/12/19 1:52 AM, Istvan Toth wrote: > > > >>>> Hi! > > > >>>> > > > >>>> I'd like to start a conversation about supporting HBase 2.2. in > the > > > >>>> master branch. > > > >>>> > > > >>>> https://issues.apache.org/jira/browse/PHOENIX-5268 has a slightly > > out > > > >> of > > > >>>> date, but functional PR for HBase 2.2 support on master. (Please > > > review > > > >>>> and comment if you have the time, I'll try to update the PR in the > > > next > > > >>>> few days) > > > >>>> > > > >>>> The reason that it is not a straightforward decision to merge it > is > > > >> that > > > >>>> applying that patch breaks compatibility with HBase 2.0.1, the > > current > > > >>>> base. > > > >>>> > > > >>>> I can see the following outcomes: > > > >>>> > > > >>>> - Do nothing > > > >>>> - Move master to HBase 2.2.2 > > > >>>> - Fork master to Hbase-2.0 and Hbase-2.2 branches > > > >>>> - Build time compatibility modules > > > >>>> - Run time compatibility modules > > > >>>> - Something that I haven't thought of > > > >>>> > > > >>>> > > > >>>> Doing nothing is obviously not a long term solution, as the > current > > > >>>> master doesn't work with any of the currently supported HBase > > > branches, > > > >>>> but we may postpone the inevitable. > > > >>>> > > > >>>> Simply moving master to HBase 2.2 is the most attractive solution > > from > > > >> a > > > >>>> pure developer POV, but there may be other considerations. > > > >>>> > > > >>>> Having multiple masters for 2.0 and 2.2 is simple from a code > > > >>>> perspective, but maintaining two branches is a non-trivial amount > of > > > >>>> additional work. (See the 4.x situation) > > > >>>> > > > >>>> Moving the HBase version dependent stuff into a separate module, > and > > > >>>> choosing at build time is not pretty from a code POV, but saves us > > the > > > >>>> hassle of maintaining multiple branches, while maintaining > > > >> compatibility > > > >>>> with multiple HBase versions, and can handle future API changes > as > > > >> well > > > >>>> from a single branch. Doing something like this could have saved > us > > > the > > > >>>> effort of maintaining three separate 4.x branches. > > > >>>> > > > >>>> I feel that since Phoenix is closely timed to HBase, and requires > > > >>>> cluster-wide HBase configuration to work anyway, handling the > > > different > > > >>>> HBase versions from the same binary/JAR is not worth the effort. > > > >>>> > > > >>>> Please share your thoughts! > > > >>>> > > > >>>> regards > > > >>>> Istvan > > > >>>> > > > >>> > > > >> > > > > > > > > > > > > > > > > -- > Best regards, > Andrew > > Words like orphans lost among the crosstalk, meaning torn from truth's > decrepit hands > - A23, Crosstalk > -- *István Tóth* | Sr. Software Engineer t. (36) 70 283-1788 st...@cloudera.com <https://www.cloudera.com> [image: Cloudera] <https://www.cloudera.com/> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera> <https://www.cloudera.com/> ------------------------------
