Great Why is nobody using it?
Best regards, Vladimir Rodionov Principal Platform Engineer Carrier IQ, www.carrieriq.com e-mail: [email protected] ________________________________________ From: Ted Yu [[email protected]] Sent: Wednesday, October 26, 2011 4:13 PM To: [email protected] Subject: Re: Random I/O performance >> Off-heap cache is experimental in 0.92 and TRUNK. As of now, TestSlabCache passes consistently in 0.92 and TRUNK. Li Pi's slides from Aug can be found here: https://docs.google.com/present/view?id=d23xkzr_55hgnvngf6 Toward the end of it, you can find performance chart. On Wed, Oct 26, 2011 at 3:49 PM, Stack <[email protected]> wrote: > On Wed, Oct 26, 2011 at 2:50 PM, Vladimir Rodionov > <[email protected]> wrote: > >>> Are you hitting cache at all? > >> > >> Its totally random, due to the proposed key design which favored fast > inserts. Keys are randomized > >> values, that is why there is no data locality in row look ups. Effect of > the cache (LruBlockCache?) is negligible > >> in this case. > >> > > > >>>So a different schema would get cache into the mix? > > > > You can/t change schema while system is in production > > > > True but caveat Ted's note and FB fellas apparently did it three times > before they hit on the 'right' schema (Not sure whether they took the > portion being modified offline when changing schema) > > > > >>>Its going to keep growing without bound? > > > > > > No, we keep data for XX days than purge stale data from the table. > > > > > > My question was: what else besides obvious -run all in parallel - can > help to improve random I/O? > > > > 1. Will BLOOM filter help to optimize HBase Read path? > > Yes. 0.92 blooms will be less expensive than those in 0.90 (because > the blooms are tiered and live in the LRU in 0.92 so they are let go > if unused). > > > > 2. We use compression already. > > 3. Block size - does it really matter much? > > Not much in my experience. Smaller blocks can help a little at the > cost of some bloat in index size (Again 0.92 is better here because > indices are partitioned and now also are in the LRU rather than pegged > in RAM as they are in 0.90). > > > 4. Off heap block cache? Its in 92 trunk? Have anybody performed real > performance tests on Off heap cache? > > > > Off-heap cache is experimental in 0.92 and TRUNK. > > St.Ack > Confidentiality Notice: The information contained in this message, including any attachments hereto, may be confidential and is intended to be read only by the individual or entity to whom this message is addressed. If the reader of this message is not the intended recipient or an agent or designee of the intended recipient, please note that any review, use, disclosure or distribution of this message or its attachments, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and/or [email protected] and delete or destroy any copy of this message and its attachments.
