A solution sounds great! Thanks for working on that.
By compile-time, I assume that means, unless we get lucky with
compatible bytecode, we'd have to have a Phoenix which is compiled
against a specific HBase version?
On 1/13/20 2:25 AM, István Tóth wrote:
Hi!
(Sorry for not replying earlier, I'll need to revisit my mail filters)
I do have a half-baked patch with the lightweight isolation in modules case.
In its current form it's more ugly than complex, but some ugliness is a
small price to pay for not having to maintain multiple branches. (And it
can be beutified later)
I will complete it, and make the PR for that as time permits.
The tephra solution handles the version differences during run (startup)
time, my current solution is compile-time.
Istvan
On Fri, Dec 20, 2019 at 11:05 AM la...@apache.org <la...@apache.org> wrote:
Yep that.
For the record... I do not think this is simple. But it is possible.
On Thursday, December 19, 2019, 8:37:37 PM GMT+1, Andrew Purtell <
apurt...@apache.org> wrote:
I can't answer for Lars but whenever version incompatibilities come up
usually only a handful of files are impacted. In the last round, the
Phoenix access controller, a related file in the same directory, and the
RPC scheduler. If you cloned these into separate version specific maven
modules case by case as needed at each round the differences are fairly
small. On the other hand if you take a principled approach and abstract all
the things, it will be a huge effort and nobody realistically will want to
take it on.
On Thu, Dec 19, 2019 at 11:34 AM Geoffrey Jacoby <gjac...@gmail.com>
wrote:
Lars,
I'm curious why you say the differences are easily isolated -- many of
the
core classes of Phoenix either directly inherit HBase classes or
implement
HBase interfaces, and those can vary between minor versions. (See my
above
example of a new coprocessor hook on BaseRegionObserver.)
Geoffrey
On Thu, Dec 19, 2019 at 10:54 AM la...@apache.org <la...@apache.org>
wrote:
Yep. The differences are pretty minimal - provided they can be
isolated
easily.
Tephra might be a pretty good model. It supports various versions of
HBase
in a single branch and has similar issues as Phoenix (coprocessors,
etc).
-- Lars
On Thursday, December 19, 2019, 7:07:51 PM GMT+1, Josh Elser <
els...@apache.org> wrote:
To clarify, you think that compat modules are better than that
separate-branches model in 4.x?
On 12/18/19 11:29 AM, la...@apache.org wrote:
This is really hard to follow.
I think we should do the same with HBase dependencies in Phoenix that
HBase does with Hadoop dependencies.
That is: We could have a maven module with the specific HBase
version
dependent code.
Btw. Tephra does the same... A module for HBase version specific
code.
-- Lars
On Tuesday, December 17, 2019, 10:00:31 AM GMT+1, Istvan Toth <
st...@apache.org> wrote:
What do you think about tying the minor releases to Hbase minor
releases
(not necessarily one-to-one)
for example (provided 5.1 is 2020H1)
5.0.0 -> HB 2.0
5.1.0 -> HB 2.2.2 (and whatever 2.1 is API compatible with it)
5.1.x -> HB 2.2.x (treat as maintenance branch, no major new
features)
5.2.0 -> HB 2.3.0 (if released by that time)
5.2.x -> HB 2.3.x (treat as maintenance branch, no major new
features)
5.3.0 -> HB 2.3.x (if there is no new major/minor Hbase release)
master -> latest released HBase version
Alternatively, we could stick with the same HBase version for patch
releases that we used for the first minor release.
This would limit the number of branches that we have to maintain in
parallel, while providing maintenance branches for older releases,
and
timely-ish Phoenix releases.
The drawback is that users of old HBase versions won't get the latest
features, on the other hand they can expect more polish.
Istvan
On Thu, Dec 12, 2019 at 8:05 PM Geoffrey Jacoby <gjac...@apache.org>
wrote:
Since HBase 2.0 is EOM'ed, I'm +1 for not worrying about 2.0.x
compatibility with the 5.x branch going forward.
Given how coupled Phoenix is to the implementation details of HBase
though,
I'm not sure trying to abstract those away to keep one Phoenix
branch
per
HBase major version is practical, however. At the least, it would be
really
complex.
For example, in the new year I plan to return to working on the
change
data
capture and Phoenix-level replication features, both of which depend
on
WALKey interface changes and a new RegionObserver coprocessor hook
introduced in HBASE-22622 and HBASE-22623. This was released in
HBase
1.5
and will be in the forthcoming HBase 2.3. While the HBase community
is
discussing EOMing 1.3 right now, and maybe 1.4 will go in the medium
term,
I don't see all pre-2.3 branch-2's getting deprecated anytime soon.
So there will be at least two significant features that can only
exist
in
some but not all of our 4.x and 5.x branches.
Geoffrey
On Thu, Dec 12, 2019 at 8:21 AM Josh Elser <els...@apache.org>
wrote:
As much as possible, I'd like to avoid us getting into another
situation
with 5.x where we have multiple branches. My hope was/is that we
can
keep one Phoenix5 branch that works against an acceptable set of
HBase
branches.
To me, that acceptable set of HBase branches is _a_ 2.1 and 2.2
release.
I don't think we need to support all 2.1.x or 2.2.x, nor do I think
we
need to keep trying to maintain 2.0.x as it's already end of
support
by
the HBase community.
Thanks for updating your PR. I'll add this to my review queue.
On 12/12/19 1:52 AM, Istvan Toth wrote:
Hi!
I'd like to start a conversation about supporting HBase 2.2. in
the
master branch.
https://issues.apache.org/jira/browse/PHOENIX-5268 has a slightly
out
of
date, but functional PR for HBase 2.2 support on master. (Please
review
and comment if you have the time, I'll try to update the PR in the
next
few days)
The reason that it is not a straightforward decision to merge it
is
that
applying that patch breaks compatibility with HBase 2.0.1, the
current
base.
I can see the following outcomes:
- Do nothing
- Move master to HBase 2.2.2
- Fork master to Hbase-2.0 and Hbase-2.2 branches
- Build time compatibility modules
- Run time compatibility modules
- Something that I haven't thought of
Doing nothing is obviously not a long term solution, as the
current
master doesn't work with any of the currently supported HBase
branches,
but we may postpone the inevitable.
Simply moving master to HBase 2.2 is the most attractive solution
from
a
pure developer POV, but there may be other considerations.
Having multiple masters for 2.0 and 2.2 is simple from a code
perspective, but maintaining two branches is a non-trivial amount
of
additional work. (See the 4.x situation)
Moving the HBase version dependent stuff into a separate module,
and
choosing at build time is not pretty from a code POV, but saves us
the
hassle of maintaining multiple branches, while maintaining
compatibility
with multiple HBase versions, and can handle future API changes
as
well
from a single branch. Doing something like this could have saved
us
the
effort of maintaining three separate 4.x branches.
I feel that since Phoenix is closely timed to HBase, and requires
cluster-wide HBase configuration to work anyway, handling the
different
HBase versions from the same binary/JAR is not worth the effort.
Please share your thoughts!
regards
Istvan
--
Best regards,
Andrew
Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
- A23, Crosstalk
