On Sat, Aug 10, 2019 at 5:58 PM Andrew Purtell <[email protected]> wrote:
> .... And yet it is also paranoia in this instance because we are talking > specifically of WALkey annotations made ONCE and BEFORE commit. Warnings > are duly noted and acknowledged! ACK the instance of paranoia and the receipt of warning. S > > > > On Aug 10, 2019, at 4:50 PM, Sean Busbey <[email protected]> wrote: > > > > How much of that trouble around debug is a matter of us not doing enough > to > > make clear when change has happened due to a coprocessor? > > > > Heck it's hard enough just saying what coprocessors were active over a > time > > period on a running cluster. If we improved that would heavy coprocessor > > use be less concerning? > > > > > > For example, coprocessor actions could go into the audit log. > > > > Or in the case of the addition of coprocessor provided metadata about the > > wal* we could track information about what added said metadata right in > the > > wall itself. > > > > > > > >> On Sat, Aug 10, 2019, 18:29 Stack <[email protected]> wrote: > >> > >>> On Fri, Aug 9, 2019 at 3:57 PM Geoffrey Jacoby <[email protected]> > wrote: > >>> > >>> > >>> You can't say "a mutable WALEdit is too dangerous" without making > >>> assumptions about the developer it's too dangerous _for_. Ideally these > >>> assumptions should be explicit and agreed upon by the community. I > think > >>> differing audience assumptions are where at least some of the > >> disagreements > >>> are coming from. > >>> > >> > >> All sounds good but I have a bit of trouble with your using WALEdit in > your > >> illustration above. > >> > >> To be explicit, the reluctance exposing WAL{Edit,Key} is because when > the > >> system goes awry, from experience, debug is consuming. Errors in WAL* > >> accounting are particularly hard to fathom because the problem shows up > >> usually at a displacement in time and location (replay or on other end > of a > >> replication). Often the issue triggers only when large amounts of data. > All > >> WAL*s flow together whatever the source out the one firehose. The > >> *assumption* is that a CP writer randomly mangling a WAL* that happens > to > >> break the cluster's recovery may not be around or game for the fun debug > >> sessions required figuring root cause. > >> > >> <joke>We could add a Geoffrey Jacoby annotation on CP APIs?</joke> > >> > >> Thanks, > >> S > >> > >> > >> > >> > >> Geoffrey > >>> > >>> > >>> On Fri, Aug 9, 2019 at 8:55 AM Andrew Purtell < > [email protected]> > >>> wrote: > >>> > >>>> The future of the coprocessor API is an interesting topic. I think we > >>> have > >>>> a range of opinions in the community and I would like to hear more of > >>> them. > >>>> > >>>> Because of the compatibility headaches sometimes I’d like to rip them > >>> out. > >>>> Sometimes they are essential for accomplishing something in Phoenix or > >>> our > >>>> in house backup solution, for example. For me my opinion is very > mixed. > >>>> > >>>> When first conceived the first use case was security and we took > >>>> inspiration from OS kernel examples in Linux and TrustedBSD (now > merged > >>>> with FreeBSD) where upcall interfaces were made available where > >>>> authoritative access control decisions are made. The set of hooks has > >>>> expanded over time as users have requested extensions. This style of > >>>> interface is powerful in that it enables a mixin approach to > >> composition > >>> of > >>>> functionality and extension. However in retrospect the maintenance > >>> burdens > >>>> were not fully appreciated, at least by me. I had assumed we would be > >>>> allowed more freedom to change but my experiences with Phoenix > educated > >>> me > >>>> on the kind of downstream headaches that result when we make those > >>> changes. > >>>> > >>>> If I were to do it again I would attempt an abstract and fluent > >> interface > >>>> where extensions would register intents and receive callbacks in a > much > >>>> more granular way, like: > >>>> > >>>> onRegion().onRPC().onGet().then(...) > >>>> > >>>> I suppose this looks kind of like Mockito. There would be no overlarge > >>>> interfaces full of upcall methods that break source compatibility on > >>> every > >>>> change (in branch-1). Although we could not avoid the complexity of > >>>> ensuring the right callbacks are invoked at the right places on the > >> right > >>>> code paths the extension interface and its types would be decoupled > >> from > >>>> internals and the kind of compatibility headaches we impose on > >>>> downstreamers (and ourselves) would mostly disappear. This has been > >>>> proposed on an old JIRA somewhere... > >>>> > >>>> Of course a redo like this would be a very complex and time consuming > >>>> project, and a port of say something like Phoenix would be a reboot of > >>>> multi man year efforts, and as far as I know nobody working in the > code > >>>> today has that kind of sustained available time and attention. It’s > too > >>>> late. We have to make the best of the legacy of past engineering > >> choices. > >>>> If that is not correct then it would be a very pleasant surprise > >> indeed. > >>>> > >>>> Given that we have the current model and we have downstreamers like > >>>> Phoenix depending on them I think we are limited in the kind of clean > >> ups > >>>> we might like to do and need to be tolerant of the requirement to > >>> maintain > >>>> these interfaces and their functionally across major versions. > >>> Deprecation > >>>> is fine but if it is done without the input of the known consumer are > >> we > >>>> really being fair? Unclear. > >>>> > >>>> > >>>>> On Aug 8, 2019, at 6:39 PM, 张铎(Duo Zhang) <[email protected]> > >>> wrote: > >>>>> > >>>>> When releasing 2.0.0 we faced a lot of problems because we exposes so > >>>> many > >>>>> internal classes to CPs. It is really hard to both consider the > >>>>> compatibility and development on HBase. And then we have done lots of > >>>> works > >>>>> to abstract interfaces for CPs to use and hide the actual > >>> implementation > >>>>> classes to be HBase only. > >>>>> > >>>>> The work is not fully done, so we still left some methods which > >> exposes > >>>>> internal classes there with a deprecated annotation, I think most of > >>> them > >>>>> are for Phoenix. This is a trade off and I think it is also > >> acceptable. > >>>>> Phoenix could still use the deprecated methods, and we will not > >> remove > >>>> them > >>>>> unless we find an alternate solution. And I think if anyone wants to > >>>> remove > >>>>> them without replacment you will first jump out and give a -1 :) > >>>>> > >>>>> Specific to HBASE-22623, I still think we should add the deprecated > >>>>> annotation to keep the API consistent, otherwise users will be > >> confused > >>>>> that whether they can use the WALEdit. And on the abstraction of > >>>> WALEdit, I > >>>>> think we used to rely on a high level abstraction in HBASE-20952. But > >>> now > >>>>> since there is little progress there, I think we can start the work > >> of > >>>>> abstracting WALEdit only. > >>>>> > >>>>> Thanks. > >>>>> > >>>>> Andrew Purtell <[email protected]> 于2019年8月9日周五 上午3:21写道： > >>>>> > >>>>>> Please let me direct your attention to the tail of HBASE-22623 for a > >>>> larger > >>>>>> discussion. I tried to sum it up as follows: > >>>>>> > >>>>>> An opinion that we should have more and more coprocessor interfaces > >> to > >>>>>> address new use cases is valid. An opinion that coprocessors are too > >>>>>> invasive and should be 'cleaned up' is also valid. An opinion that > >> the > >>>>>> compatibility headaches of coprocessor interfaces are annoying is > >>>> valid. An > >>>>>> opinion that Phoenix can be considered as a valid use case when > >>>> considering > >>>>>> interface changes is valid. An opinion that only HBase level > >> concerns > >>>>>> should motivate API changes is valid. These opinions are strawmen. I > >>>> think > >>>>>> they approach actual positions in the community but I do not imply > >> any > >>>>>> specific person has one of them. These strawmen are at least > >> partially > >>>>>> contradictory. It is going to be an ongoing process to sort them out > >>>> into > >>>>>> something that makes sense and can get consensus. > >>>>>> > >>>>>> So while as committer I am moving forward on HBASE-22623 because I > >>> don't > >>>>>> see a veto but instead a disagreement on the margins (deprecation or > >>>> not) > >>>>>> motivated on larger principles, I also want to raise the visibility > >> of > >>>> the > >>>>>> disagreement because I think it impacts our relationship with > >> another > >>>>>> project at Apache at a minimum, but also future technical directions > >>> of > >>>> an > >>>>>> important subset of interfaces. > >>>>>> > >>>>>> For your consideration. > >>>>>> > >>>>>> -- > >>>>>> Best regards, > >>>>>> Andrew > >>>>>> > >>>>>> Words like orphans lost among the crosstalk, meaning torn from > >> truth's > >>>>>> decrepit hands > >>>>>> - A23, Crosstalk > >>>>>> > >>>> > >>> > >> >
