StoreScanner gives 3x degradation on sequential scans with PREFIX_TREE enabled. Block size 64K. StoreScanner has a lot of overhead including but not limited to KeyValueHeap and ScanQueryMatcher.
On Sat, Oct 19, 2013 at 8:33 PM, Ted Yu <[email protected]> wrote: > Interesting. > > From this graph (HBASE-4676), PREFIX_TREE block encoding is not sensitive > to variation in block size: > > https://issues.apache.org/jira/secure/attachment/12500778/SeeksPerSec%20by%20blockSize.png > > What was your block size ? > If you don't bypass StoreScanner/KeyValueHeap, would that make a difference > ? > > Thanks > > > On Sat, Oct 19, 2013 at 7:34 PM, Vladimir Rodionov > <[email protected]>wrote: > > > What I wanted to say by this? HBase still does not have block encoding > > which is optimal for both scan and seek (re-seek). > > I do not think these goals are mutually exclusive. > > > > > > Best regards, > > Vladimir Rodionov > > Principal Platform Engineer > > Carrier IQ, www.carrieriq.com > > e-mail: [email protected] > > > > ________________________________________ > > From: Vladimir Rodionov [[email protected]] > > Sent: Saturday, October 19, 2013 7:32 PM > > To: [email protected] > > Subject: Beware of PREFIX_TREE block encoding > > > > The scan performance is bad. 10 x slower on my tests than for blocks with > > NONE encoding. I scan data directly from block cache through > > StoreFileScanner (bypassing all StoreScanner/KeyValueHeap stuff). It > should > > be clearly stated that this encoding degrades overall performance > > significantly in favor of data size reduction and is suitable only for > Gets > > - not for Scans. > > > > Best regards, > > -Vladimir Rodionov > > > > - > > > > Confidentiality Notice: The information contained in this message, > > including any attachments hereto, may be confidential and is intended to > be > > read only by the individual or entity to whom this message is addressed. > If > > the reader of this message is not the intended recipient or an agent or > > designee of the intended recipient, please note that any review, use, > > disclosure or distribution of this message or its attachments, in any > form, > > is strictly prohibited. If you have received this message in error, > please > > immediately notify the sender and/or [email protected] and > > delete or destroy any copy of this message and its attachments. > > >
