Re: HBase 2 slower than HBase 1?

2020-12-01 Thread ramkrishna vasudevan
Hi Bruno Regarding the perf issue with more CPs you might be interested in seeing this https://issues.apache.org/jira/browse/HBASE-25277 Regards Ram On Thu, Nov 5, 2020 at 11:25 PM ramkrishna vasudevan < ramkrishna.s.vasude...@gmail.com> wrote: > This is great information. Thanks Bruno. Keep

Re: HBase 2 slower than HBase 1?

2020-11-05 Thread ramkrishna vasudevan
This is great information. Thanks Bruno. Keep us posted if you find anything more when you have your isolated tests done. Regards Ram On Thu, Nov 5, 2020 at 10:36 PM Bruno Dumon wrote: > Hi Ram, > > Sorry I missed your question. Yes, we do use addColumns. > > Specifically, we also query the

Re: HBase 2 slower than HBase 1?

2020-11-05 Thread Bruno Dumon
Hi Ram, Sorry I missed your question. Yes, we do use addColumns. Specifically, we also query the Phoenix existence marker, which is a column named "_0", which is typically the last column in the row (we query this through custom scans, not through Phoenix). It does seem that reading the last

Re: HBase 2 slower than HBase 1?

2020-10-30 Thread ramkrishna vasudevan
Hi Bruno/Jan Just a query here. I read your emails in this thread. One simple question just to ensure if your tests were similar to Andrew's test. Did your scan query have addColumns explicitly added covering all (or most of) the columns in the rows? Regards Ram On Wed, Jul 15, 2020 at 7:02

Re: HBase 2 slower than HBase 1?

2020-07-14 Thread Andrew Purtell
This is a good finding, nice work! I added a comment on HBASE-24742 that mentions HBASE-24637 on the off chance they are related, although I suspect more changes are implicated by the 2.x regression. On Tue, Jul 14, 2020 at 5:53 PM Bharath Vissapragada wrote: > FYI, we filed this today

Re: HBase 2 slower than HBase 1?

2020-07-14 Thread Andrew Purtell
I went out on vacation (and am still out) before tracking this down. If you are waiting for me to make more progress with HBASE-24637, I can do that in a couple of weeks, Anyone is welcome to step in sooner. .. On Tue, Jul 14, 2020 at 11:38 AM Josh Elser wrote: > Wow. Great stuff, Andrew! > >

Re: HBase 2 slower than HBase 1?

2020-07-14 Thread Bharath Vissapragada
FYI, we filed this today https://issues.apache.org/jira/browse/HBASE-24742. We ran into a similar regression when upgrading from 1.3 based branch to 1.6 based branch. After some profiling and code analysis we narrowed down the code paths. On Tue, Jul 14, 2020 at 11:38 AM Josh Elser wrote: >

Re: HBase 2 slower than HBase 1?

2020-07-14 Thread Josh Elser
Wow. Great stuff, Andrew! Thank you for compiling and posting it all here. I can only imagine how time-consuming this was. On 6/26/20 1:57 PM, Andrew Purtell wrote: Hey Anoop, I opened https://issues.apache.org/jira/browse/HBASE-24637 and attached the patches and script used to make the

Re: HBase 2 slower than HBase 1?

2020-06-26 Thread Andrew Purtell
Hey Anoop, I opened https://issues.apache.org/jira/browse/HBASE-24637 and attached the patches and script used to make the comparison. On Fri, Jun 26, 2020 at 2:33 AM Anoop John wrote: > Great investigation Andy. Do you know any Jiras which made changes in SQM? > Would be great if you can

Re: HBase 2 slower than HBase 1?

2020-06-26 Thread Anoop John
Great investigation Andy. Do you know any Jiras which made changes in SQM? Would be great if you can attach your patch which tracks the scan flow. If we have a Jira for this issue, can you pls attach? Anoop On Fri, Jun 26, 2020 at 1:56 AM Andrew Purtell wrote: > Related, I think I found a

Re: HBase 2 slower than HBase 1?

2020-06-25 Thread Andrew Purtell
Related, I think I found a bug in branch-1 where we don’t heartbeat in the filter all case until we switch store files, so scanning a very large store file might time out with client defaults. Remarking on this here so I don’t forget to follow up. > On Jun 25, 2020, at 12:27 PM, Andrew

Re: HBase 2 slower than HBase 1?

2020-06-25 Thread Andrew Purtell
I repeated this test with pe --filterAll and the results were revealing, at least for this case. I also patched in thread local hash map for atomic counters that I could update from code paths in SQM, StoreScanner, HFileReader*, and HFileBlock. Because a RPC is processed by a single handler thread