I saw beta == no new features, no API changes, just fixes. And I am working on HBASE-18805 to unify Admin and AsyncAdmin methods. The fix version was 2.0-beta-1. But I thought this will introduce API change(deprecate some API and introduce new one). So should I change the fix versions to 2.0-alpha-4 and finish it before we release 2.0-alpha-4?
Issue HBASE-18978 (Align the methods in Table and AsyncTable) may have same problem. Thanks. 2017-10-26 9:51 GMT+08:00 张铎(Duo Zhang) <palomino...@gmail.com>: > OK, skimmed, we are in trouble! The in memory compaction just use the same > constructor with normal compaction to construct a StoreScanner, and use it > to do compaction... > > We have to provide several preXXX and postXXX for it, at least we should > allow user reset TTL and max versions, and also do filtering on the > scanner. > > Thanks. > > > > 2017-10-26 9:41 GMT+08:00 张铎(Duo Zhang) <palomino...@gmail.com>: > > > When adding back the pre(Flush/Compact/Store)ScannerOpen methods in > > HBASE-19033, I found that there maybe a hole in our CP hools. It is the > in > > memory compaction. > > > > As Anoop suggested in HBASE-19001, we still need to give user the ability > > to extend the max versions and TTL config, so I plan to add back the > hooks > > above to let CP users can change the max versions and TTL of a ScanInfo > > object. But I'm not sure whether in memory compaction will also discard > > expired cells, if so then we are in trouble... > > > > Thanks. > > > > 2017-10-25 6:03 GMT+08:00 Stack <st...@duboce.net>: > > > >> Chatting with my coworker Mr. Mike Drob, we were batting back and forth > >> what remains to be done. Surfacing our thoughts here so you all clued > >> in....Or if you think otherwise, please speak up. > >> > >> We have ~13 issues to land: > >> https://issues.apache.org/jira/projects/HBASE/versions/12341594 About > two > >> are meta-issues that are about process which leaves 11. > >> > >> Duo and Zheng Hu are to merge the FilterList fixes improvements > >> (HBASE-19057, HBASE-18410 et al.). These are blocker because some > changed > >> API/semantic that we need to get out earlier rather than later. > >> > >> Once the above is merged, HBASE-13346, change of Filter method names to > >> mention Cell instead of KeyValue can land. > >> > >> HBASE-199048 needs a review (Anoop will probably do it), removing > >> IA.Private objects as params to MasterObserver... Hopefully this goes in > >> soon. > >> > >> Duo is hard at work on trackers for flush and compaction for CPs > >> (HBASE-18905). How is HBASE-19033 looking Duo (facility for Tephra)? > >> > >> I think HBASE-18906 (Phoenix Region#waitFor...) will evaporate after Duo > >> is > >> done w/ his work above. > >> > >> I'm on HBASE-18770 bypass and HBASE-19077 restore some parity after all > >> the > >> purges allowing CPs do direct calls against Regions in same Host. > >> > >> Anoop is on HBASE-19047 (Fixes) and Ram on cleanup of CellUtil. > >> > >> Another day or two? > >> > >> St.Ack > >> > >> > >> > >> On Mon, Oct 23, 2017 at 2:52 PM, Stack <st...@duboce.net> wrote: > >> > >> > > >> > On Mon, Oct 23, 2017 at 11:59 AM, Josh Elser <els...@apache.org> > wrote: > >> > > >> >> +1 > >> >> > >> >> I was trying to work on helping out on the outstanding alpha-4 stuff > >> last > >> >> week -- will be continuing to try to do the same this week. > >> >> > >> >> If you need any help, Stack, or if others need reviews where I > haven't > >> >> noticed on my own: feel free to @mention me. > >> >> > >> >> > >> > Thanks for the offer Josh. All items seem assigned and are being > >> actively > >> > worked on. If you get a moment, reviews by you (or anyone else) helps > >> move > >> > the process along. > >> > > >> > We need to merge in HBASE-18410 branch to pick up Filter improvements. > >> > Then HBASE-13346 can go in. > >> > > >> > You are already helping out on HBASE-18906, thanks. Looks like that > will > >> > be addressed by other alpha-4s about to land. > >> > > >> > St.Ack > >> > TODOs: https://issues.apache.org/jira/projects/HBASE/versions/ > 12341594 > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> >> On 10/23/17 12:53 PM, Stack wrote: > >> >> > >> >>> (Reviving this thread) > >> >>> > >> >>> Lets push out alpha-4 this week. Alpha-4 is the release that has the > >> >>> refactor of the Coprocessor API shutting down access to internals > >> marked > >> >>> InterfaceAudience.Private. > >> >>> > >> >>> The outstanding list is here: > >> >>> https://issues.apache.org/jira/projects/HBASE/versions/12341594 > >> >>> > >> >>> Please push in anything marked alpha-4 that belongs to you. > >> >>> > >> >>> If issue, talk out loud on this thread. If you need a review to land > >> an > >> >>> item, shout on the issue and here; we'll help you out. > >> >>> > >> >>> As is, items are coming along nicely I'd say. We need to merge the > >> filter > >> >>> branch -- HBASE-18410 -- so APIs are finished for hbase2. > >> >>> > >> >>> Post alpha-4, we'll have to hunt down our downstreamers and help > them > >> >>> test > >> >>> on top of alpha-4 so rolling into beta-1, we have confidence our > >> >>> downstreamers know what to expect (or we discover what we missed > >> BEFORE > >> >>> we > >> >>> beta-1). > >> >>> > >> >>> Thanks for time, > >> >>> S > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> On Fri, Sep 8, 2017 at 2:04 PM, Stack <st...@duboce.net> wrote: > >> >>> > >> >>> I'll put up an alpha3 RC Monday, probably Monday night. That should > be > >> >>>> time, if we all sprint, for the public-facing API fixes to be done. > >> >>>> > >> >>>> I had a bunch of Coprocessor refactor and fixup scheduled for > alpha3 > >> but > >> >>>> it is plain that more time is needed (in spite of valiant effort so > >> far > >> >>>> by > >> >>>> Anoop, Duo, Appy, etc.). Therefore, lets run a 2.0.0-alpha-4 whose > >> >>>> theme is > >> >>>> "Coprocessor Fixup". Hopefully we can put an alpha-4 up by the > >> following > >> >>>> week. > >> >>>> > >> >>>> We should then be ready for beta (beta == no new features, no API > >> >>>> changes, > >> >>>> just fixes). > >> >>>> > >> >>>> Thanks, > >> >>>> St.Ack > >> >>>> > >> >>>> > >> >>>> On Thu, Aug 17, 2017 at 12:35 PM, Stack <st...@duboce.net> wrote: > >> >>>> > >> >>>> I put up the hbase-2.0.0-alpha2 release candidate. Please vote on > it. > >> >>>>> > >> >>>>> For hbase-2.0.0-alpha3, the theme is solidifying API. I hope to > get > >> a > >> >>>>> release out in the next week or so. > >> >>>>> > >> >>>>> I did a weeding of 2.0.0 issues over the last day. If folks are > >> >>>>> interested in helping out, below are the items I think we need > done > >> for > >> >>>>> alpha3 (below are at least 'Critical' status, are API possibly > >> altering > >> >>>>> items, and are absent those JIRAs that are making active progress, > >> >>>>> i.e. the > >> >>>>> HTD/HCD revamp by Chia-Ping Tsai). A project NOT listed that needs > >> >>>>> doing is > >> >>>>> what Andrew did comparing 1.3. and 1.4 APIs > >> >>>>> > >> >>>>> * HBASE-18622 Mitigate compatibility concerns between branch-1 and > >> >>>>> branch-2 > >> >>>>> This is to do what Andrew did between 1.3 and 1.4 branches only do > >> it > >> >>>>> between branch-1 and branch-2. > >> >>>>> > >> >>>>> * HBASE-10462 Recategorize some of the client facing Public / > >> Private > >> >>>>> interfaces > >> >>>>> This one is almost done. It could do with a finish, attention to > the > >> >>>>> items in last comment, and then our codebase could do with another > >> >>>>> sweep > >> >>>>> after the spirit of this issue since a bunch has gone in since the > >> pass > >> >>>>> that was the basis of this issue. > >> >>>>> > >> >>>>> * HBASE-10504 Define Replication Interface > >> >>>>> I was going to take a crack at this as part of the revamp forced > by > >> >>>>> 'HBASE-15982 Interface ReplicationEndpoint extends Guava's > Service' > >> >>>>> but if > >> >>>>> anyone else is interested, be my guest. > >> >>>>> > >> >>>>> * HBASE-14996 Some more API cleanup for 2.0 > >> >>>>> Has a bunch of subtasks, some of which are being worked on. Needs > >> >>>>> finishing. > >> >>>>> > >> >>>>> * HBASE-14998 Unify synchronous and asynchronous methods in Admin > >> and > >> >>>>> cleanup > >> >>>>> Needs a pass. Small issue I think. Could also look at new > >> AsyncClient > >> >>>>> and > >> >>>>> make sure symmetry. > >> >>>>> > >> >>>>> * HBASE-15607 Remove PB references from Admin for 2.0 > >> >>>>> Predicated on result of an ongoing DISCUSSION thread but needs to > be > >> >>>>> done. > >> >>>>> > >> >>>>> Rolling upgrade will have implications for our API. Would be good > to > >> >>>>> try > >> >>>>> it and figure what needs fixup (as said above, according to trial > by > >> >>>>> Sean, > >> >>>>> we might not be too bad here): > >> >>>>> * HBASE-16060 1.x clients cannot access table state talking to 2.0 > >> >>>>> cluster > >> >>>>> * HBASE-16550 Procedure v2 - Add AM compatibility for 2.x Master > and > >> >>>>> 1.x > >> >>>>> RSs; i.e. support Rolling Upgrade from hbase-1 to -2. > >> >>>>> > >> >>>>> * HBASE-17442 Move most of the replication related classes to > >> >>>>> hbase-server package > >> >>>>> The above would be good to do generally but it may make for > ripples > >> in > >> >>>>> API so would be good to do now. > >> >>>>> > >> >>>>> * HBASE-18106 Redo ProcedureInfo and LockInfo > >> >>>>> Balazs is working on this. The idea is that we avoid adding two > new > >> >>>>> types > >> >>>>> to our API, two types that are nought but curtailed, read-only > >> views on > >> >>>>> internals. Input if you have time appreciated. > >> >>>>> > >> >>>>> * HBASE-18596 A hbase1 cluster should be able to replicate to a > >> hbase2 > >> >>>>> cluster; verify > >> >>>>> Esteban is looking at this one > >> >>>>> > >> >>>>> * HBASE-9417 SecureBulkLoadEndpoint should be folded in core > >> >>>>> * HBASE-17143 Scan improvement > >> >>>>> > >> >>>>> Our Coprocessor Interface needs a tough edit. It exposes > >> >>>>> implementations > >> >>>>> marked audience Private and returns implementations rather than > >> >>>>> Interfaces. > >> >>>>> In a few locations, we allow returning an alternate implementation > >> >>>>> altogether which is probably something we don't want a CP doing. > To > >> >>>>> that > >> >>>>> end, the following issues started by Duo and Anoop need to be > taken > >> to > >> >>>>> the > >> >>>>> finish line; ideally they'd have an owner: > >> >>>>> > >> >>>>> * HBASE-18169 Coprocessor fix and cleanup before 2.0.0 release <= > >> The > >> >>>>> umbrella issue. > >> >>>>> * HBASE-18298 RegionServerServices Interface cleanup for CP expose > >> >>>>> * HBASE-16769 Deprecate/remove PB references from MasterObserver > and > >> >>>>> RegionServerObserver > >> >>>>> > >> >>>>> > >> >>>>> Nice-to-haves: > >> >>>>> > >> >>>>> * HBASE-15284 Make TimeRange constructors IA.Private and remove > >> unused > >> >>>>> TimeRange constructors > >> >>>>> > >> >>>>> * HBASE-10944 Remove all kv.getBuffer() and kv.getRow() references > >> >>>>> existing in the code > >> >>>>> This is the end of an old long-running project moving up on to > Cell > >> >>>>> Interface. We think it is done but for a few little items > >> (deprecate KV > >> >>>>> methods in MR and provide Cell versions instead...) > >> >>>>> > >> >>>>> * HBASE-13271 Table#puts(List<Put>) operation is indeterminate; > >> needs > >> >>>>> fixing > >> >>>>> > >> >>>>> * HBASE-13346 Clean up Filter package for post 1.0 > >> >>>>> > >> >>>>> * HBASE-14255 Simplify Cell creation post 1.0 > >> >>>>> * HBASE-14997 > >> >>>>> Move compareOp and Comparators out of filter to client package > >> >>>>> > >> >>>>> * HBASE-13740 Stop using Hadoop private interfaces > >> >>>>> > >> >>>>> What about: > >> >>>>> > >> >>>>> * HBASE-18601 Remove Htrace 3.2 > >> >>>>> As has been noted, the HTrace API is our 'trace' API. > >> >>>>> > >> >>>>> If interested in any of the above and you need a legup, just ask > in > >> the > >> >>>>> issue and I'll be by.... > >> >>>>> > >> >>>>> Thanks, > >> >>>>> St.Ack > >> >>>>> > >> >>>>> > >> >>>>> > >> >>>>> On Mon, Aug 14, 2017 at 10:54 AM, Stack <st...@duboce.net> wrote: > >> >>>>> > >> >>>>> Heads-up: > >> >>>>>> > >> >>>>>> I'm about to put up an hbase-2.0.0-alpha2 Release Candidate. > Theme > >> is > >> >>>>>> updated dependencies, reliance on relocated popular libs (guava, > >> >>>>>> netty, > >> >>>>>> protobuf), purge of checked-in generated src, and > >> >>>>>> master-carries-no-regions > >> >>>>>> by default. > >> >>>>>> > >> >>>>>> alpha3 I hope will follow soon after (end-of-August?). Its theme > >> will > >> >>>>>> be > >> >>>>>> settling the APIs and compatibility (At first blush, we are not > >> >>>>>> looking too > >> >>>>>> bad; our Sean ran some tests over weekend that have hbase-1 > client > >> >>>>>> running > >> >>>>>> against an hbase-2 cluster....). The Coprocessor Interface revamp > >> >>>>>> should be > >> >>>>>> done by alpha3 (i.e. returning Interfaces rather than > >> >>>>>> Implementations, and > >> >>>>>> our shutdown of CPs accessing classes in hbase marked > >> >>>>>> InterfaceAudience). > >> >>>>>> We'll also have purged thirdparty classes from our API; e.g. > guava > >> >>>>>> 0.12 > >> >>>>>> Service showing through in our replication API and protobufs in > >> Admin > >> >>>>>> Interface. On alpha3, we will have to do a bunch of outreach to > >> make > >> >>>>>> sure > >> >>>>>> our downstreamers are up on what is coming down the pipe. > >> >>>>>> > >> >>>>>> Beta1 in mid-September? > >> >>>>>> > >> >>>>>> I encourage you to check out the items marked for hbase2: > >> >>>>>> https://issues.apache.org/jira/projects/HBASE/versions/12327188 > >> Edit > >> >>>>>> as > >> >>>>>> you see appropriate. Punt if you know the JIRA will not get any > >> >>>>>> attention > >> >>>>>> in next month or so. > >> >>>>>> > >> >>>>>> A bunch of issues marked blocker are unassigned. I'll leave them > >> as is > >> >>>>>> another while but I'll boot them soon. > >> >>>>>> > >> >>>>>> While I have your attention: > >> >>>>>> > >> >>>>>> + I think we should leave thrift version at 0.9.3. Moving hbase > >> thrift > >> >>>>>> to 0.10.0 will break existing clients. The change is easy enough > if > >> >>>>>> folks > >> >>>>>> need to upgrade their hbase thrift. See HBASE-18591. > >> >>>>>> + Upgrade from 0.94 is disallowed. You have to get to 1.0 first > >> >>>>>> (0.98?). > >> >>>>>> > >> >>>>>> St.Ack > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> On Wed, Aug 2, 2017 at 9:43 AM, Stack <st...@duboce.net> wrote: > >> >>>>>> > >> >>>>>> > >> >>>>>>> > >> >>>>>>> On Tue, Aug 1, 2017 at 2:06 PM, Josh Elser <els...@apache.org> > >> >>>>>>> wrote: > >> >>>>>>> > >> >>>>>>> > >> >>>>>>>> > >> >>>>>>>> On 7/31/17 9:00 AM, Stack wrote: > >> >>>>>>>> > >> >>>>>>>> On Mon, Jul 24, 2017 at 12:25 PM, Josh Elser<els...@apache.org > > > >> >>>>>>>>> wrote: > >> >>>>>>>>> > >> >>>>>>>>> ... > >> >>>>>>>>> > >> >>>>>>>>>> > >> >>>>>>>>>> I like the idea of this also hitting 2.0 as it would make the > >> >>>>>>>>>> feature a > >> >>>>>>>>>> bit more "real", but am obviously a little nervous (I have no > >> >>>>>>>>>> reason > >> >>>>>>>>>> to be > >> >>>>>>>>>> nervous though). I am pretty happy with the feature in terms > of > >> >>>>>>>>>> how > >> >>>>>>>>>> much it > >> >>>>>>>>>> is covered via testing. > >> >>>>>>>>>> > >> >>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-17748 > >> >>>>>>>>>> > >> >>>>>>>>>> > >> >>>>>>>>>> > >> >>>>>>>>>> Sounds good to me. Whats involved? Backport? If so, +1 Josh. > >> >>>>>>>>>> > >> >>>>>>>>> > >> >>>>>>>>> Last think on space quota says that need doc too. See 'Space > >> >>>>>>>>> Quota' in > >> >>>>>>>>> here: > >> >>>>>>>>> https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9i > >> >>>>>>>>> Eu_ktczrlKHK8N4SZzs/edit#heading=h.wuw3a6jukzo5 > >> >>>>>>>>> Does this little section need an update Josh? > >> >>>>>>>>> > >> >>>>>>>>> Thanks, > >> >>>>>>>>> S > >> >>>>>>>>> > >> >>>>>>>>> > >> >>>>>>>> Yep, just a couple of cherry-picks. Good test coverage and some > >> docs > >> >>>>>>>> already included for 17748. Happy to put that on my plate if > >> >>>>>>>> you're good > >> >>>>>>>> with it. I can reasonably assume that no one is against it :) > >> >>>>>>>> > >> >>>>>>>> I think I had knocked out docs for the "phase 1" stuff before > we > >> >>>>>>>> merged it in from the original feature branch. I'll double > check > >> >>>>>>>> and update > >> >>>>>>>> the gdoc. Perhaps this was just a timing thing. > >> >>>>>>>> > >> >>>>>>>> > >> >>>>>>> Thanks Josh, > >> >>>>>>> S > >> >>>>>>> > >> >>>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>> > >> >>>> > >> >>> > >> > > >> > > > > >
