The future of the coprocessor API is an interesting topic. I think we have a
range of opinions in the community and I would like to hear more of them.
Because of the compatibility headaches sometimes I’d like to rip them out.
Sometimes they are essential for accomplishing something in Phoenix or our in
house backup solution, for example. For me my opinion is very mixed.
When first conceived the first use case was security and we took inspiration
from OS kernel examples in Linux and TrustedBSD (now merged with FreeBSD) where
upcall interfaces were made available where authoritative access control
decisions are made. The set of hooks has expanded over time as users have
requested extensions. This style of interface is powerful in that it enables a
mixin approach to composition of functionality and extension. However in
retrospect the maintenance burdens were not fully appreciated, at least by me.
I had assumed we would be allowed more freedom to change but my experiences
with Phoenix educated me on the kind of downstream headaches that result when
we make those changes.
If I were to do it again I would attempt an abstract and fluent interface where
extensions would register intents and receive callbacks in a much more granular
way, like:
onRegion().onRPC().onGet().then(...)
I suppose this looks kind of like Mockito. There would be no overlarge
interfaces full of upcall methods that break source compatibility on every
change (in branch-1). Although we could not avoid the complexity of ensuring
the right callbacks are invoked at the right places on the right code paths the
extension interface and its types would be decoupled from internals and the
kind of compatibility headaches we impose on downstreamers (and ourselves)
would mostly disappear. This has been proposed on an old JIRA somewhere...
Of course a redo like this would be a very complex and time consuming project,
and a port of say something like Phoenix would be a reboot of multi man year
efforts, and as far as I know nobody working in the code today has that kind of
sustained available time and attention. It’s too late. We have to make the best
of the legacy of past engineering choices. If that is not correct then it would
be a very pleasant surprise indeed.
Given that we have the current model and we have downstreamers like Phoenix
depending on them I think we are limited in the kind of clean ups we might like
to do and need to be tolerant of the requirement to maintain these interfaces
and their functionally across major versions. Deprecation is fine but if it is
done without the input of the known consumer are we really being fair? Unclear.
> On Aug 8, 2019, at 6:39 PM, 张铎(Duo Zhang) <[email protected]> wrote:
>
> When releasing 2.0.0 we faced a lot of problems because we exposes so many
> internal classes to CPs. It is really hard to both consider the
> compatibility and development on HBase. And then we have done lots of works
> to abstract interfaces for CPs to use and hide the actual implementation
> classes to be HBase only.
>
> The work is not fully done, so we still left some methods which exposes
> internal classes there with a deprecated annotation, I think most of them
> are for Phoenix. This is a trade off and I think it is also acceptable.
> Phoenix could still use the deprecated methods, and we will not remove them
> unless we find an alternate solution. And I think if anyone wants to remove
> them without replacment you will first jump out and give a -1 :)
>
> Specific to HBASE-22623, I still think we should add the deprecated
> annotation to keep the API consistent, otherwise users will be confused
> that whether they can use the WALEdit. And on the abstraction of WALEdit, I
> think we used to rely on a high level abstraction in HBASE-20952. But now
> since there is little progress there, I think we can start the work of
> abstracting WALEdit only.
>
> Thanks.
>
> Andrew Purtell <[email protected]> 于2019年8月9日周五 上午3:21写道：
>
>> Please let me direct your attention to the tail of HBASE-22623 for a larger
>> discussion. I tried to sum it up as follows:
>>
>> An opinion that we should have more and more coprocessor interfaces to
>> address new use cases is valid. An opinion that coprocessors are too
>> invasive and should be 'cleaned up' is also valid. An opinion that the
>> compatibility headaches of coprocessor interfaces are annoying is valid. An
>> opinion that Phoenix can be considered as a valid use case when considering
>> interface changes is valid. An opinion that only HBase level concerns
>> should motivate API changes is valid. These opinions are strawmen. I think
>> they approach actual positions in the community but I do not imply any
>> specific person has one of them. These strawmen are at least partially
>> contradictory. It is going to be an ongoing process to sort them out into
>> something that makes sense and can get consensus.
>>
>> So while as committer I am moving forward on HBASE-22623 because I don't
>> see a veto but instead a disagreement on the margins (deprecation or not)
>> motivated on larger principles, I also want to raise the visibility of the
>> disagreement because I think it impacts our relationship with another
>> project at Apache at a minimum, but also future technical directions of an
>> important subset of interfaces.
>>
>> For your consideration.
>>
>> --
>> Best regards,
>> Andrew
>>
>> Words like orphans lost among the crosstalk, meaning torn from truth's
>> decrepit hands
>> - A23, Crosstalk
>>
