HBASE-22577 is almost there. But I think I still need one more +1 here so I can merge HBASE-21512 back...
张铎(Duo Zhang) <palomino...@gmail.com> 于2019年6月14日周五 下午4:54写道： > After applying HBASE-22577 the performance is good enough now. And the > YCSB workloada also shows that there are no big differences on performance. > Please see the comments on HBASE-22564 for more details. > > Will get HBASE-22577 in soon, and resolve the pending issues such > as HBASE-22237. > > Any other concerns? Thanks. > > 张铎(Duo Zhang) <palomino...@gmail.com> 于2019年6月14日周五 上午11:27写道： > >> It’s hadoop-2.8.5. >> >> But I do not think it makes much difference, we just use the same server >> setup, only different clients implementation. >> >> Sakthi <sakthivel.azh...@gmail.com>于2019年6月14日 周五10:18写道： >> >>> Duo, >>> >>> What version on Hadoop did you use in the 5 node cluster for your >>> comparisons? For what it's worth, I would also like to try out the PE/LTT >>> comparison of both the clients. Or, if any other form of comparison would >>> be helpful then I'm open for suggestions and would like to give it a try. >>> >>> Sakthi >>> >>> On Thu, Jun 13, 2019 at 4:55 PM 张铎(Duo Zhang) <palomino...@gmail.com> >>> wrote: >>> >>> > It will be transparent, mostly, the interface is still kept. There are >>> some >>> > incompatible behaviors, for example, now admin.split will wait till the >>> > split is actually done, while in the old time it will return >>> immediately >>> > after we send the request to master. >>> > >>> > Jean-Marc Spaggiari <jean-m...@spaggiari.org>于2019年6月13日 周四23:26写道： >>> > >>> > > Hi, >>> > > >>> > > Is this going to change the way the client should be called? Or it >>> will >>> > be >>> > > mostly transparent replacement? >>> > > >>> > > Thanks, >>> > > >>> > > JMS >>> > > >>> > > Le jeu. 13 juin 2019 à 02:13, 张铎(Duo Zhang) <palomino...@gmail.com> >>> a >>> > > écrit : >>> > > >>> > > > Josh Elser <els...@apache.org> 于2019年6月12日周三 下午10:00写道： >>> > > > >>> > > > > Nice perf results! >>> > > > > >>> > > > > https://issues.apache.org/jira/browse/HBASE-22237 looks like >>> it's >>> > also >>> > > > > good to be resolved, given >>> > > > > >>> > > > > >>> > > > >>> > > >>> > >>> https://builds.apache.org/job/HBASE%20Nightly/job/HBASE-21512/279/testReport/ >>> > > > > (TestLogLevel will be fixed on your rebase/merge). >>> > > > > >>> > > > > Poking through the PR, it looks like the big change is that we're >>> > also >>> > > > > defaulting over to use the [sync]ConnectionOverAsyncConnection. >>> Good >>> > to >>> > > > > do it now to help iron things out more. Calling it out to make >>> sure >>> > > > > others see this. Is it still possible to use the old Connection >>> impl? >>> > > (I >>> > > > > think the answer is "no"). >>> > > > > >>> > > > No, all the code have been purged... >>> > > > >>> > > > > >>> > > > > Only other question: are there updates for the book that should >>> > happen >>> > > > > before you move past this? What about "knobs" for configuring >>> > retries, >>> > > > > internal thread pool(s)? Anything like that you think would be >>> > > important >>> > > > > for people to tweak? >>> > > > > >>> > > > Will fill a 'fat' release note soon. I think there will be less >>> > > parameters >>> > > > to tune, as we do not need any thread pools unless you are using >>> > > > coprocessor related methods(which are deprecated and we recommend >>> users >>> > > to >>> > > > use the ones in async client interface). The retry config is still >>> the >>> > > same >>> > > > with the old sync client. >>> > > > >>> > > > > >>> > > > > +1 >>> > > > > >>> > > > > On 6/11/19 5:48 AM, 张铎(Duo Zhang) wrote: >>> > > > > > Filed https://issues.apache.org/jira/browse/HBASE-22564 >>> > > > > > >>> > > > > > 张铎(Duo Zhang) <palomino...@gmail.com> 于2019年6月11日周二 下午3:53写道： >>> > > > > > >>> > > > > >> Let me do a YCSB test about the performance. >>> > > > > >> >>> > > > > >> Stack <st...@duboce.net> 于2019年6月11日周二 下午1:15写道： >>> > > > > >> >>> > > > > >>> +1 on merge from me. >>> > > > > >>> >>> > > > > >>> It removes the complicated multi-threaded edifice we'd built >>> > > > > client-side >>> > > > > >>> to >>> > > > > >>> fake an async behavior replacing it with an actual async >>> > > > > implementation. >>> > > > > >>> Users will immediately notice a radical plummet in working >>> thread >>> > > > > count on >>> > > > > >>> the client side. >>> > > > > >>> >>> > > > > >>> For the cleanup of old idioms alone, in test code in >>> particular, >>> > > the >>> > > > > patch >>> > > > > >>> is worth merging. >>> > > > > >>> >>> > > > > >>> Any perf numbers to share comparing old sync and async? >>> > > > > >>> >>> > > > > >>> What about difference in operation? Is there any commentary >>> or >>> > doc >>> > > or >>> > > > > >>> release note to point at? >>> > > > > >>> >>> > > > > >>> Thanks, >>> > > > > >>> S >>> > > > > >>> >>> > > > > >>> >>> > > > > >>> >>> > > > > >>> On Mon, Jun 10, 2019 at 6:59 PM 张铎(Duo Zhang) < >>> > > palomino...@gmail.com >>> > > > > >>> > > > > >>> wrote: >>> > > > > >>> >>> > > > > >>>> https://issues.apache.org/jira/browse/HBASE-21512 >>> > > > > >>>> >>> > > > > >>>> "Reimplement sync client based on async client" >>> > > > > >>>> >>> > > > > >>>> The jira title tells everything. This is what I promised >>> when I >>> > > > first >>> > > > > >>>> introduced the async client in HBase, about three years ago, >>> > that >>> > > > the >>> > > > > >>> sync >>> > > > > >>>> client can be implemented on top of the async client, so we >>> can >>> > > > remove >>> > > > > >>> the >>> > > > > >>>> old sync client implementation, which can reduce our client >>> code >>> > > > base >>> > > > > a >>> > > > > >>>> lot. >>> > > > > >>>> >>> > > > > >>>> I've already opened a PR here, and received several >>> > > feedback(thanks >>> > > > > >>> stack!) >>> > > > > >>>> >>> > > > > >>>> https://github.com/apache/hbase/pull/287 >>> > > > > >>>> >>> > > > > >>>> It shows that we add 8,663 lines and remove 31,386 lines. >>> > > > > >>>> >>> > > > > >>>> This is the flaky dashboard for this branch >>> > > > > >>>> >>> > > > > >>>> >>> > > > > >>>> >>> > > > > >>> >>> > > > > >>> > > > >>> > > >>> > >>> https://builds.apache.org/job/HBASE-Find-Flaky-Tests/job/HBASE-21512/lastSuccessfulBuild/artifact/dashboard.html >>> > > > > >>>> >>> > > > > >>>> With the recent efforts I think it is getting better. >>> > > > > >>>> >>> > > > > >>>> Will fill the release note soon, it will be a fat one. >>> > > > > >>>> >>> > > > > >>>> Please vote >>> > > > > >>>> >>> > > > > >>>> [] +1 >>> > > > > >>>> [] +0/-0 >>> > > > > >>>> [] -1 Do not merge the branch back because ... >>> > > > > >>>> >>> > > > > >>>> Thanks. Any suggestions are welcomed. >>> > > > > >>>> >>> > > > > >>> >>> > > > > >> >>> > > > > > >>> > > > > >>> > > > >>> > > >>> > >>> >>
