I'd also like to get "Splittable Meta" HBASE-12233 in. Apart from scalability this would help spread the load on meta access across a bunch of servers. We have a branch-1.3 implementation, I'll put it up got sidetracked a bit when I was cleaning it up. Then after that a trunk patch.
Thanks, Francis On Wed, May 8, 2019 at 10:50 PM Yu Li <[email protected]> wrote: > Thanks for the feedback guys, and good to know your thoughts/requirements > about ccsmap, will start scheduling for it. > > I believe in-memory flush/compaction could enhance the performance, the > only concern for production is stability. Would be great if we could see it > earlier on production like in 2.3/2.4 > > Best Regards, > Yu > > > On Wed, 8 May 2019 at 15:13, Anoop John <[email protected]> wrote: > > > > I do not think these two features can be done in 3.0. > > Fine.. I was just asking as we said cloud readiness. Fully agree to the > > concern. Might be good for 4.0 > > > > Agree to other items in the list. Ya lets try to stabilize the > Compacting > > Memstore feature. Am ready to help in that area. > > > > Anoop > > > > On Mon, May 6, 2019 at 11:05 PM Andrew Purtell <[email protected]> > > wrote: > > > > > Also +1 for ccsmap and this could even be useful and of reasonable risk > > to > > > be backported to a 1.6.0 (if we ever make one) > > > > > > On Mon, May 6, 2019 at 9:20 AM 张洸豪 <[email protected]> wrote: > > > > > > > +1 for ccsmap. > > > > For in memory flush/compaction, @ZhengH tested it once. The gc time > > > > reduced a lot and p99/p999 is small and stable. We will try it on our > > > > cluster. So I thought this feature may be production ready in HBase > 2.3 > > > or > > > > 2.4? > > > > Another problem is about scalability. Now we have one production > > cluster > > > > which have thousands tables and hundreds of thousands regions. There > > are > > > > more than 50000 Qps to meta region. We have to disable client > prefetch > > to > > > > reduce the pressure of meta. So we should reconsider to enable more > > than > > > > one meta region for HBase 3.0. > > > > > > > > > > > > > > > > ---Original--- > > > > From: "Sean Busbey"<[email protected]> > > > > Date: Mon, May 6, 2019 21:06 PM > > > > To: "dev"<[email protected]>; > > > > Subject: Re: [DISCUSS] HBase 3 > > > > > > > > > > > > I agree with Duo, neither of those is close to done enough for 3.0. > > > > > > > > On Mon, May 6, 2019, 06:48 张铎(Duo Zhang) <[email protected]> > > wrote: > > > > > > > > > I do not think these two features can be done in 3.0... > > > > > > > > > > Anoop John <[email protected]>于2019年5月6日 周一14:48写道： > > > > > > > > > > > For the cloud usages, we should include the new FS abstraction > > issue > > > > also > > > > > > targeted ? What abt the WAL on Ratis? We can fix the features > > what > > > we > > > > > > would like to see in 3.0 and then have feature freeze. So that > 3.0 > > > wont > > > > > > become so huge changes. > > > > > > > > > > > > Anoop > > > > > > > > > > > > On Mon, May 6, 2019 at 9:52 AM Yu Li <[email protected]> wrote: > > > > > > > > > > > > > TL;DR: Maybe firstly we should figure out what direction we > > should > > > > > focus > > > > > > > for 3.0? It seems to me current performance is good enough for > > most > > > > use > > > > > > > case and no longer a big concern, so more requirements are > about > > > > > > stability > > > > > > > and elasticity (in cloud)? Or if anyone do have performance > > > concern, > > > > > > please > > > > > > > shout and let us know, thanks. > > > > > > > > > > > > > > All below items are performance oriented, just list out for > > > reference > > > > > > (and > > > > > > > not sure about priority from community perspective): > > > > > > > 1. CCSMap (HBASE-20312) (delayed due to job priority, sorry, > but > > we > > > > > could > > > > > > > get it done if required). > > > > > > > 2. Maybe we should also target at making > > in-memory-flush/compaction > > > > > from > > > > > > > experimental to production ready? > > > > > > > 3. Server-side asynchronous (especially the write pipeline) is > > > > another > > > > > > > left-over job I ever promised to upstream but failed because > the > > > > > business > > > > > > > growth on my side slows down quite a bit so it's not fully > > verified > > > > > > online > > > > > > > yet. > > > > > > > > > > > > > > Best Regards, > > > > > > > Yu > > > > > > > > > > > > > > > > > > > > > On Mon, 6 May 2019 at 06:07, Andrew Purtell < > [email protected] > > > > > > > > wrote: > > > > > > > > > > > > > > > Definitely there is value in starting this so trunk doesn't > > sink > > > > > into a > > > > > > > > difficult to release state. Alpha releases seem fine if you > > want > > > to > > > > > > make > > > > > > > > them. This assumes you'll have at least a small amount of > > > bandwidth > > > > > to > > > > > > > run > > > > > > > > tests and triage any issues, though, or else any effort would > > be > > > > > better > > > > > > > > spent completing the work needed to move the stable pointer > to > > > 2.x. > > > > > > Just > > > > > > > my > > > > > > > > random advice. > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Apr 25, 2019 at 7:55 AM Sean Busbey < > [email protected] > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Hi folks! > > > > > > > > > > > > > > > > > > We're about a year from when HBase 2.0.0 went GA. I'd like > to > > > > start > > > > > > > > > cutting alpha releases of 3.0.0 off of the master branch > > soon. > > > > > > > > > (expressly I do not want to create another branch, so for > now > > > > > > anything > > > > > > > > > landing in master would be "in" for 3.0.0. hence the > "alpha" > > > > > > > > > designation.) > > > > > > > > > > > > > > > > > > I was going to wait for one of the HBase 2 releases to get > > the > > > > > stable > > > > > > > > > label. But lately I don't have a good sense of if that will > > > > happen > > > > > > > > > within a month or within six months, so I'm leaning away > from > > > > > > waiting. > > > > > > > > > > > > > > > > > > I personally don't have a particular feature I'm trying to > > get > > > > out > > > > > > the > > > > > > > > > door. I just think we're at risk of another > waiting-too-long > > > for > > > > a > > > > > > > > > major release and want to get started on the work of > > > quantifying > > > > > > > > > what's changed and figuring out how downstream projects are > > > > > impacted. > > > > > > > > > I find that all much easier to do when there's a release > > > artifact > > > > > to > > > > > > > > > reference. > > > > > > > > > > > > > > > > > > I'm happy to just cut alpha releases until someone shows up > > > with > > > > a > > > > > > > > > specific feature need. At that point we can come up with > > > criteria > > > > > for > > > > > > > > > entering and exiting beta releases. > > > > > > > > > > > > > > > > > > What do folks plan to work on getting ready that needs > happen > > > in > > > > a > > > > > > > > > major release? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Best regards, > > > > > > > > Andrew > > > > > > > > > > > > > > > > Words like orphans lost among the crosstalk, meaning torn > from > > > > > truth's > > > > > > > > decrepit hands > > > > > > > > - A23, Crosstalk > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Best regards, > > > Andrew > > > > > > Words like orphans lost among the crosstalk, meaning torn from truth's > > > decrepit hands > > > - A23, Crosstalk > > > > > >
