> > For public API we have rules on how to keep compatible so I think it is > less hurt for users, beta1 is fine. > Got it. Thanks for you explanation.
When you think you'd be done w/ the sync of Admin and AsyncAdmin > Interfaces? Are the changes to Admin or to AsyncAdmin? There are two issues (HBASE-19009 and HBASE-18950) need more review. After resolve these issues, I will upload a new patch for HBASE-18911. Thanks. 2017-10-27 0:14 GMT+08:00 Stack <st...@duboce.net>: > On Thu, Oct 26, 2017 at 2:27 AM, Guanghao Zhang <zghao...@gmail.com> > wrote: > > > I saw beta == no new features, no API changes, just fixes. And I am > working > > on HBASE-18805 to unify Admin and AsyncAdmin methods. The fix version was > > 2.0-beta-1. But I thought this will introduce API change(deprecate some > API > > and introduce new one). So should I change the fix versions to > 2.0-alpha-4 > > and finish it before we release 2.0-alpha-4? > > > > Issue HBASE-18978 (Align the methods in Table and AsyncTable) may have > same > > problem. Thanks. > > > > > Yeah, we were supposed to be done w/ public API changes when we shipped > alpha-3 (smile). > > For Admin Interface, semantic versioning applies as per Duo comment. I see > some polish still being done on the Admin/Table APIs (removal of deprecated > methods, alignment of Admin and AsyncAdmin APIs). Hard to say no to this > clean up on our most public facing Interfaces. > > When you think you'd be done w/ the sync of Admin and AsyncAdmin > Interfaces? Are the changes to Admin or to AsyncAdmin? > > Thank you Guanghao Zhang, > > St.Ack > > > > 2017-10-26 9:51 GMT+08:00 张铎(Duo Zhang) <palomino...@gmail.com>: > > > > > OK, skimmed, we are in trouble! The in memory compaction just use the > > same > > > constructor with normal compaction to construct a StoreScanner, and use > > it > > > to do compaction... > > > > > > We have to provide several preXXX and postXXX for it, at least we > should > > > allow user reset TTL and max versions, and also do filtering on the > > > scanner. > > > > > > Thanks. > > > > > > > > > > > > 2017-10-26 9:41 GMT+08:00 张铎(Duo Zhang) <palomino...@gmail.com>: > > > > > > > When adding back the pre(Flush/Compact/Store)ScannerOpen methods in > > > > HBASE-19033, I found that there maybe a hole in our CP hools. It is > the > > > in > > > > memory compaction. > > > > > > > > As Anoop suggested in HBASE-19001, we still need to give user the > > ability > > > > to extend the max versions and TTL config, so I plan to add back the > > > hooks > > > > above to let CP users can change the max versions and TTL of a > ScanInfo > > > > object. But I'm not sure whether in memory compaction will also > discard > > > > expired cells, if so then we are in trouble... > > > > > > > > Thanks. > > > > > > > > 2017-10-25 6:03 GMT+08:00 Stack <st...@duboce.net>: > > > > > > > >> Chatting with my coworker Mr. Mike Drob, we were batting back and > > forth > > > >> what remains to be done. Surfacing our thoughts here so you all > clued > > > >> in....Or if you think otherwise, please speak up. > > > >> > > > >> We have ~13 issues to land: > > > >> https://issues.apache.org/jira/projects/HBASE/versions/12341594 > About > > > two > > > >> are meta-issues that are about process which leaves 11. > > > >> > > > >> Duo and Zheng Hu are to merge the FilterList fixes improvements > > > >> (HBASE-19057, HBASE-18410 et al.). These are blocker because some > > > changed > > > >> API/semantic that we need to get out earlier rather than later. > > > >> > > > >> Once the above is merged, HBASE-13346, change of Filter method names > > to > > > >> mention Cell instead of KeyValue can land. > > > >> > > > >> HBASE-199048 needs a review (Anoop will probably do it), removing > > > >> IA.Private objects as params to MasterObserver... Hopefully this > goes > > in > > > >> soon. > > > >> > > > >> Duo is hard at work on trackers for flush and compaction for CPs > > > >> (HBASE-18905). How is HBASE-19033 looking Duo (facility for Tephra)? > > > >> > > > >> I think HBASE-18906 (Phoenix Region#waitFor...) will evaporate after > > Duo > > > >> is > > > >> done w/ his work above. > > > >> > > > >> I'm on HBASE-18770 bypass and HBASE-19077 restore some parity after > > all > > > >> the > > > >> purges allowing CPs do direct calls against Regions in same Host. > > > >> > > > >> Anoop is on HBASE-19047 (Fixes) and Ram on cleanup of CellUtil. > > > >> > > > >> Another day or two? > > > >> > > > >> St.Ack > > > >> > > > >> > > > >> > > > >> On Mon, Oct 23, 2017 at 2:52 PM, Stack <st...@duboce.net> wrote: > > > >> > > > >> > > > > >> > On Mon, Oct 23, 2017 at 11:59 AM, Josh Elser <els...@apache.org> > > > wrote: > > > >> > > > > >> >> +1 > > > >> >> > > > >> >> I was trying to work on helping out on the outstanding alpha-4 > > stuff > > > >> last > > > >> >> week -- will be continuing to try to do the same this week. > > > >> >> > > > >> >> If you need any help, Stack, or if others need reviews where I > > > haven't > > > >> >> noticed on my own: feel free to @mention me. > > > >> >> > > > >> >> > > > >> > Thanks for the offer Josh. All items seem assigned and are being > > > >> actively > > > >> > worked on. If you get a moment, reviews by you (or anyone else) > > helps > > > >> move > > > >> > the process along. > > > >> > > > > >> > We need to merge in HBASE-18410 branch to pick up Filter > > improvements. > > > >> > Then HBASE-13346 can go in. > > > >> > > > > >> > You are already helping out on HBASE-18906, thanks. Looks like > that > > > will > > > >> > be addressed by other alpha-4s about to land. > > > >> > > > > >> > St.Ack > > > >> > TODOs: https://issues.apache.org/jira/projects/HBASE/versions/ > > > 12341594 > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> >> On 10/23/17 12:53 PM, Stack wrote: > > > >> >> > > > >> >>> (Reviving this thread) > > > >> >>> > > > >> >>> Lets push out alpha-4 this week. Alpha-4 is the release that has > > the > > > >> >>> refactor of the Coprocessor API shutting down access to > internals > > > >> marked > > > >> >>> InterfaceAudience.Private. > > > >> >>> > > > >> >>> The outstanding list is here: > > > >> >>> https://issues.apache.org/jira/projects/HBASE/versions/12341594 > > > >> >>> > > > >> >>> Please push in anything marked alpha-4 that belongs to you. > > > >> >>> > > > >> >>> If issue, talk out loud on this thread. If you need a review to > > land > > > >> an > > > >> >>> item, shout on the issue and here; we'll help you out. > > > >> >>> > > > >> >>> As is, items are coming along nicely I'd say. We need to merge > the > > > >> filter > > > >> >>> branch -- HBASE-18410 -- so APIs are finished for hbase2. > > > >> >>> > > > >> >>> Post alpha-4, we'll have to hunt down our downstreamers and help > > > them > > > >> >>> test > > > >> >>> on top of alpha-4 so rolling into beta-1, we have confidence our > > > >> >>> downstreamers know what to expect (or we discover what we missed > > > >> BEFORE > > > >> >>> we > > > >> >>> beta-1). > > > >> >>> > > > >> >>> Thanks for time, > > > >> >>> S > > > >> >>> > > > >> >>> > > > >> >>> > > > >> >>> > > > >> >>> > > > >> >>> On Fri, Sep 8, 2017 at 2:04 PM, Stack <st...@duboce.net> wrote: > > > >> >>> > > > >> >>> I'll put up an alpha3 RC Monday, probably Monday night. That > > should > > > be > > > >> >>>> time, if we all sprint, for the public-facing API fixes to be > > done. > > > >> >>>> > > > >> >>>> I had a bunch of Coprocessor refactor and fixup scheduled for > > > alpha3 > > > >> but > > > >> >>>> it is plain that more time is needed (in spite of valiant > effort > > so > > > >> far > > > >> >>>> by > > > >> >>>> Anoop, Duo, Appy, etc.). Therefore, lets run a 2.0.0-alpha-4 > > whose > > > >> >>>> theme is > > > >> >>>> "Coprocessor Fixup". Hopefully we can put an alpha-4 up by the > > > >> following > > > >> >>>> week. > > > >> >>>> > > > >> >>>> We should then be ready for beta (beta == no new features, no > API > > > >> >>>> changes, > > > >> >>>> just fixes). > > > >> >>>> > > > >> >>>> Thanks, > > > >> >>>> St.Ack > > > >> >>>> > > > >> >>>> > > > >> >>>> On Thu, Aug 17, 2017 at 12:35 PM, Stack <st...@duboce.net> > > wrote: > > > >> >>>> > > > >> >>>> I put up the hbase-2.0.0-alpha2 release candidate. Please vote > on > > > it. > > > >> >>>>> > > > >> >>>>> For hbase-2.0.0-alpha3, the theme is solidifying API. I hope > to > > > get > > > >> a > > > >> >>>>> release out in the next week or so. > > > >> >>>>> > > > >> >>>>> I did a weeding of 2.0.0 issues over the last day. If folks > are > > > >> >>>>> interested in helping out, below are the items I think we need > > > done > > > >> for > > > >> >>>>> alpha3 (below are at least 'Critical' status, are API possibly > > > >> altering > > > >> >>>>> items, and are absent those JIRAs that are making active > > progress, > > > >> >>>>> i.e. the > > > >> >>>>> HTD/HCD revamp by Chia-Ping Tsai). A project NOT listed that > > needs > > > >> >>>>> doing is > > > >> >>>>> what Andrew did comparing 1.3. and 1.4 APIs > > > >> >>>>> > > > >> >>>>> * HBASE-18622 Mitigate compatibility concerns between branch-1 > > and > > > >> >>>>> branch-2 > > > >> >>>>> This is to do what Andrew did between 1.3 and 1.4 branches > only > > do > > > >> it > > > >> >>>>> between branch-1 and branch-2. > > > >> >>>>> > > > >> >>>>> * HBASE-10462 Recategorize some of the client facing Public / > > > >> Private > > > >> >>>>> interfaces > > > >> >>>>> This one is almost done. It could do with a finish, attention > to > > > the > > > >> >>>>> items in last comment, and then our codebase could do with > > another > > > >> >>>>> sweep > > > >> >>>>> after the spirit of this issue since a bunch has gone in since > > the > > > >> pass > > > >> >>>>> that was the basis of this issue. > > > >> >>>>> > > > >> >>>>> * HBASE-10504 Define Replication Interface > > > >> >>>>> I was going to take a crack at this as part of the revamp > forced > > > by > > > >> >>>>> 'HBASE-15982 Interface ReplicationEndpoint extends Guava's > > > Service' > > > >> >>>>> but if > > > >> >>>>> anyone else is interested, be my guest. > > > >> >>>>> > > > >> >>>>> * HBASE-14996 Some more API cleanup for 2.0 > > > >> >>>>> Has a bunch of subtasks, some of which are being worked on. > > Needs > > > >> >>>>> finishing. > > > >> >>>>> > > > >> >>>>> * HBASE-14998 Unify synchronous and asynchronous methods in > > Admin > > > >> and > > > >> >>>>> cleanup > > > >> >>>>> Needs a pass. Small issue I think. Could also look at new > > > >> AsyncClient > > > >> >>>>> and > > > >> >>>>> make sure symmetry. > > > >> >>>>> > > > >> >>>>> * HBASE-15607 Remove PB references from Admin for 2.0 > > > >> >>>>> Predicated on result of an ongoing DISCUSSION thread but needs > > to > > > be > > > >> >>>>> done. > > > >> >>>>> > > > >> >>>>> Rolling upgrade will have implications for our API. Would be > > good > > > to > > > >> >>>>> try > > > >> >>>>> it and figure what needs fixup (as said above, according to > > trial > > > by > > > >> >>>>> Sean, > > > >> >>>>> we might not be too bad here): > > > >> >>>>> * HBASE-16060 1.x clients cannot access table state talking to > > 2.0 > > > >> >>>>> cluster > > > >> >>>>> * HBASE-16550 Procedure v2 - Add AM compatibility for 2.x > Master > > > and > > > >> >>>>> 1.x > > > >> >>>>> RSs; i.e. support Rolling Upgrade from hbase-1 to -2. > > > >> >>>>> > > > >> >>>>> * HBASE-17442 Move most of the replication related classes to > > > >> >>>>> hbase-server package > > > >> >>>>> The above would be good to do generally but it may make for > > > ripples > > > >> in > > > >> >>>>> API so would be good to do now. > > > >> >>>>> > > > >> >>>>> * HBASE-18106 Redo ProcedureInfo and LockInfo > > > >> >>>>> Balazs is working on this. The idea is that we avoid adding > two > > > new > > > >> >>>>> types > > > >> >>>>> to our API, two types that are nought but curtailed, read-only > > > >> views on > > > >> >>>>> internals. Input if you have time appreciated. > > > >> >>>>> > > > >> >>>>> * HBASE-18596 A hbase1 cluster should be able to replicate to > a > > > >> hbase2 > > > >> >>>>> cluster; verify > > > >> >>>>> Esteban is looking at this one > > > >> >>>>> > > > >> >>>>> * HBASE-9417 SecureBulkLoadEndpoint should be folded in core > > > >> >>>>> * HBASE-17143 Scan improvement > > > >> >>>>> > > > >> >>>>> Our Coprocessor Interface needs a tough edit. It exposes > > > >> >>>>> implementations > > > >> >>>>> marked audience Private and returns implementations rather > than > > > >> >>>>> Interfaces. > > > >> >>>>> In a few locations, we allow returning an alternate > > implementation > > > >> >>>>> altogether which is probably something we don't want a CP > doing. > > > To > > > >> >>>>> that > > > >> >>>>> end, the following issues started by Duo and Anoop need to be > > > taken > > > >> to > > > >> >>>>> the > > > >> >>>>> finish line; ideally they'd have an owner: > > > >> >>>>> > > > >> >>>>> * HBASE-18169 Coprocessor fix and cleanup before 2.0.0 release > > <= > > > >> The > > > >> >>>>> umbrella issue. > > > >> >>>>> * HBASE-18298 RegionServerServices Interface cleanup for CP > > expose > > > >> >>>>> * HBASE-16769 Deprecate/remove PB references from > MasterObserver > > > and > > > >> >>>>> RegionServerObserver > > > >> >>>>> > > > >> >>>>> > > > >> >>>>> Nice-to-haves: > > > >> >>>>> > > > >> >>>>> * HBASE-15284 Make TimeRange constructors IA.Private and > remove > > > >> unused > > > >> >>>>> TimeRange constructors > > > >> >>>>> > > > >> >>>>> * HBASE-10944 Remove all kv.getBuffer() and kv.getRow() > > references > > > >> >>>>> existing in the code > > > >> >>>>> This is the end of an old long-running project moving up on to > > > Cell > > > >> >>>>> Interface. We think it is done but for a few little items > > > >> (deprecate KV > > > >> >>>>> methods in MR and provide Cell versions instead...) > > > >> >>>>> > > > >> >>>>> * HBASE-13271 Table#puts(List<Put>) operation is > indeterminate; > > > >> needs > > > >> >>>>> fixing > > > >> >>>>> > > > >> >>>>> * HBASE-13346 Clean up Filter package for post 1.0 > > > >> >>>>> > > > >> >>>>> * HBASE-14255 Simplify Cell creation post 1.0 > > > >> >>>>> * HBASE-14997 > > > >> >>>>> Move compareOp and Comparators out of filter to client package > > > >> >>>>> > > > >> >>>>> * HBASE-13740 Stop using Hadoop private interfaces > > > >> >>>>> > > > >> >>>>> What about: > > > >> >>>>> > > > >> >>>>> * HBASE-18601 Remove Htrace 3.2 > > > >> >>>>> As has been noted, the HTrace API is our 'trace' API. > > > >> >>>>> > > > >> >>>>> If interested in any of the above and you need a legup, just > ask > > > in > > > >> the > > > >> >>>>> issue and I'll be by.... > > > >> >>>>> > > > >> >>>>> Thanks, > > > >> >>>>> St.Ack > > > >> >>>>> > > > >> >>>>> > > > >> >>>>> > > > >> >>>>> On Mon, Aug 14, 2017 at 10:54 AM, Stack <st...@duboce.net> > > wrote: > > > >> >>>>> > > > >> >>>>> Heads-up: > > > >> >>>>>> > > > >> >>>>>> I'm about to put up an hbase-2.0.0-alpha2 Release Candidate. > > > Theme > > > >> is > > > >> >>>>>> updated dependencies, reliance on relocated popular libs > > (guava, > > > >> >>>>>> netty, > > > >> >>>>>> protobuf), purge of checked-in generated src, and > > > >> >>>>>> master-carries-no-regions > > > >> >>>>>> by default. > > > >> >>>>>> > > > >> >>>>>> alpha3 I hope will follow soon after (end-of-August?). Its > > theme > > > >> will > > > >> >>>>>> be > > > >> >>>>>> settling the APIs and compatibility (At first blush, we are > not > > > >> >>>>>> looking too > > > >> >>>>>> bad; our Sean ran some tests over weekend that have hbase-1 > > > client > > > >> >>>>>> running > > > >> >>>>>> against an hbase-2 cluster....). The Coprocessor Interface > > revamp > > > >> >>>>>> should be > > > >> >>>>>> done by alpha3 (i.e. returning Interfaces rather than > > > >> >>>>>> Implementations, and > > > >> >>>>>> our shutdown of CPs accessing classes in hbase marked > > > >> >>>>>> InterfaceAudience). > > > >> >>>>>> We'll also have purged thirdparty classes from our API; e.g. > > > guava > > > >> >>>>>> 0.12 > > > >> >>>>>> Service showing through in our replication API and protobufs > in > > > >> Admin > > > >> >>>>>> Interface. On alpha3, we will have to do a bunch of outreach > to > > > >> make > > > >> >>>>>> sure > > > >> >>>>>> our downstreamers are up on what is coming down the pipe. > > > >> >>>>>> > > > >> >>>>>> Beta1 in mid-September? > > > >> >>>>>> > > > >> >>>>>> I encourage you to check out the items marked for hbase2: > > > >> >>>>>> https://issues.apache.org/jira/projects/HBASE/versions/ > > 12327188 > > > >> Edit > > > >> >>>>>> as > > > >> >>>>>> you see appropriate. Punt if you know the JIRA will not get > any > > > >> >>>>>> attention > > > >> >>>>>> in next month or so. > > > >> >>>>>> > > > >> >>>>>> A bunch of issues marked blocker are unassigned. I'll leave > > them > > > >> as is > > > >> >>>>>> another while but I'll boot them soon. > > > >> >>>>>> > > > >> >>>>>> While I have your attention: > > > >> >>>>>> > > > >> >>>>>> + I think we should leave thrift version at 0.9.3. Moving > hbase > > > >> thrift > > > >> >>>>>> to 0.10.0 will break existing clients. The change is easy > > enough > > > if > > > >> >>>>>> folks > > > >> >>>>>> need to upgrade their hbase thrift. See HBASE-18591. > > > >> >>>>>> + Upgrade from 0.94 is disallowed. You have to get to 1.0 > first > > > >> >>>>>> (0.98?). > > > >> >>>>>> > > > >> >>>>>> St.Ack > > > >> >>>>>> > > > >> >>>>>> > > > >> >>>>>> > > > >> >>>>>> On Wed, Aug 2, 2017 at 9:43 AM, Stack <st...@duboce.net> > > wrote: > > > >> >>>>>> > > > >> >>>>>> > > > >> >>>>>>> > > > >> >>>>>>> On Tue, Aug 1, 2017 at 2:06 PM, Josh Elser < > els...@apache.org > > > > > > >> >>>>>>> wrote: > > > >> >>>>>>> > > > >> >>>>>>> > > > >> >>>>>>>> > > > >> >>>>>>>> On 7/31/17 9:00 AM, Stack wrote: > > > >> >>>>>>>> > > > >> >>>>>>>> On Mon, Jul 24, 2017 at 12:25 PM, Josh Elser< > > els...@apache.org > > > > > > > >> >>>>>>>>> wrote: > > > >> >>>>>>>>> > > > >> >>>>>>>>> ... > > > >> >>>>>>>>> > > > >> >>>>>>>>>> > > > >> >>>>>>>>>> I like the idea of this also hitting 2.0 as it would make > > the > > > >> >>>>>>>>>> feature a > > > >> >>>>>>>>>> bit more "real", but am obviously a little nervous (I > have > > no > > > >> >>>>>>>>>> reason > > > >> >>>>>>>>>> to be > > > >> >>>>>>>>>> nervous though). I am pretty happy with the feature in > > terms > > > of > > > >> >>>>>>>>>> how > > > >> >>>>>>>>>> much it > > > >> >>>>>>>>>> is covered via testing. > > > >> >>>>>>>>>> > > > >> >>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-17748 > > > >> >>>>>>>>>> > > > >> >>>>>>>>>> > > > >> >>>>>>>>>> > > > >> >>>>>>>>>> Sounds good to me. Whats involved? Backport? If so, +1 > > Josh. > > > >> >>>>>>>>>> > > > >> >>>>>>>>> > > > >> >>>>>>>>> Last think on space quota says that need doc too. See > 'Space > > > >> >>>>>>>>> Quota' in > > > >> >>>>>>>>> here: > > > >> >>>>>>>>> https://docs.google.com/document/d/ > > 1WCsVlnHjJeKUcl7wHwqb4z9i > > > >> >>>>>>>>> Eu_ktczrlKHK8N4SZzs/edit#heading=h.wuw3a6jukzo5 > > > >> >>>>>>>>> Does this little section need an update Josh? > > > >> >>>>>>>>> > > > >> >>>>>>>>> Thanks, > > > >> >>>>>>>>> S > > > >> >>>>>>>>> > > > >> >>>>>>>>> > > > >> >>>>>>>> Yep, just a couple of cherry-picks. Good test coverage and > > some > > > >> docs > > > >> >>>>>>>> already included for 17748. Happy to put that on my plate > if > > > >> >>>>>>>> you're good > > > >> >>>>>>>> with it. I can reasonably assume that no one is against it > :) > > > >> >>>>>>>> > > > >> >>>>>>>> I think I had knocked out docs for the "phase 1" stuff > before > > > we > > > >> >>>>>>>> merged it in from the original feature branch. I'll double > > > check > > > >> >>>>>>>> and update > > > >> >>>>>>>> the gdoc. Perhaps this was just a timing thing. > > > >> >>>>>>>> > > > >> >>>>>>>> > > > >> >>>>>>> Thanks Josh, > > > >> >>>>>>> S > > > >> >>>>>>> > > > >> >>>>>>> > > > >> >>>>>> > > > >> >>>>>> > > > >> >>>>> > > > >> >>>> > > > >> >>> > > > >> > > > > >> > > > > > > > > > > > > > >
