Created https://issues.apache.org/jira/browse/HBASE-9751.
On Fri, Oct 11, 2013 at 8:55 PM, lars hofhansl <[email protected]> wrote: > Taking a quick look... It is easy to reduce the ThreadLocal calls by 50% > without any refactoring in the Memstore. > > Also thinking about this more, we already have metadata in HFile about the > newest memstoreTS; I used that optimize storing the memstoreTS in HFiles > away (HBASE-8166 and HBASE-8151). > > We could use this during scanning as well and avoid checking against the > readpoint altogether. > (might need some refactoring, though, have to checked closer). > > > -- Lars > > > > ----- Original Message ----- > From: lars hofhansl <[email protected]> > To: "[email protected]" <[email protected]> > Cc: > Sent: Friday, October 11, 2013 6:28 PM > Subject: Re: skipKVsNewerThanReadpoint in StoreFileScanner and > MemStoreScanner > > Interesting. Never came up in my profiling of raw scan performance. > Of course my assumption was that the vast majority of all KVs are in > HFiles and thus in the blockcache (not the memstore). > > ThreadLocals are indeed expensive, we ran into such problems as well (code > unrelated to HBase, though). > > > Mind filing a jira, Vladimir? If there's a perf issue, we should fix it. > > > -- Lars > > ________________________________ > > From: Vladimir Rodionov <[email protected]> > To: "[email protected]" <[email protected]> > Sent: Friday, October 11, 2013 2:36 PM > Subject: Re: skipKVsNewerThanReadpoint in StoreFileScanner and > MemStoreScanner > > > Yes, threadLocal.get() is expensive when you count every nanosecond. There > is no need to call it every time in MemStoreScanner.next(). > > > > On Fri, Oct 11, 2013 at 2:32 PM, Ted Yu <[email protected]> wrote: > > > MultiVersionConsistencyControl.getThreadReadPoint() retrieves value from > a > > ThreadLocal. > > > > Can you specify which operation becomes expensive ? > > > > BTW I only found IsolationLevel.READ_UNCOMMITTED > > in TestRowProcessorEndpoint. Looks like it is not normally used. > > > > > > On Fri, Oct 11, 2013 at 2:20 PM, Vladimir Rodionov > > <[email protected]>wrote: > > > > > Update. > > > > > > This method is not called in MemStoreScanner, but direct call to : > > > > > > readPoint = MultiVersionConsistencyControl.getThreadReadPoint(); > > > > > > on every next, seek etc > > > which is basically the same. > > > > > > All these calls become very expensive when you try to achieve 10s of > > > millions RPS in scan operations. > > > > > > Best regards, > > > Vladimir Rodionov > > > Principal Platform Engineer > > > Carrier IQ, www.carrieriq.com > > > e-mail: [email protected] > > > > > > ________________________________________ > > > From: Vladimir Rodionov > > > Sent: Friday, October 11, 2013 2:18 PM > > > To: [email protected] > > > Subject: skipKVsNewerThanReadpoint in StoreFileScanner and > > MemStoreScanner > > > > > > From HRegion: > > > > > > synchronized(scannerReadPoints) { > > > if (isolationLevel == IsolationLevel.READ_UNCOMMITTED) { > > > // This scan can read even uncommitted transactions > > > this.readPt = Long.MAX_VALUE; > > > > MultiVersionConsistencyControl.setThreadReadPoint(this.readPt); > > > } else { > > > this.readPt = > > > MultiVersionConsistencyControl.resetThreadReadPoint(mvcc); > > > } > > > > > > If isolation level is READ_UNCOMMITTED than this method should never be > > > called in both StoreFileScanner and MemStoreScanner > > > If isolation level is READ_COMMITTED than this method must be called > only > > > in MemStoreScanner, because HFiles are immutable no new KV's can be > added > > > to HFile after StoreFileScanner was created. > > > > > > > > > Correct? > > > > > > > > > Best regards, > > > Vladimir Rodionov > > > Principal Platform Engineer > > > Carrier IQ, www.carrieriq.com > > > e-mail: [email protected] > > > > > > Confidentiality Notice: The information contained in this message, > > > including any attachments hereto, may be confidential and is intended > to > > be > > > read only by the individual or entity to whom this message is > addressed. > > If > > > the reader of this message is not the intended recipient or an agent or > > > designee of the intended recipient, please note that any review, use, > > > disclosure or distribution of this message or its attachments, in any > > form, > > > is strictly prohibited. If you have received this message in error, > > please > > > immediately notify the sender and/or [email protected] and > > > delete or destroy any copy of this message and its attachments. > > > > > >
