FS latency shouldn't matter with your 99.9% cache hit rate as reported. On Wed, Apr 20, 2011 at 12:55 PM, Dmitriy Lyubimov <[email protected]>wrote:
> Yes -- I already looked thru 'regionserver' metrics some time ago in > hbase book. And i am not sure there's a 'inner ttlb' metric. > > There are fs latency metrics there but nothing for the respons times. > fs latency is essentially hdfs latency AFAICT and that would not be > relevant to what i am asking for (for as long as we are hitting LRU > block cache anyway). we are not hitting fs. > > Unless there are more metrics than listed in the Hbase Book? > > > On Wed, Apr 20, 2011 at 12:46 PM, Stack <[email protected]> wrote: > > Enable rpc logging. Will show in your ganglia. See metrics article > > on hbase home page. > > > > On Wed, Apr 20, 2011 at 12:44 PM, Dmitriy Lyubimov <[email protected]> > wrote: > >> Is there any way to log 'inner' TTLB times the region server incurs for > reads? > >> > >> > >> On Wed, Apr 20, 2011 at 12:43 PM, Dmitriy Lyubimov <[email protected]> > wrote: > >>> i just enabled debug logging for o.a.h.hbase logger in that particular > >>> region server... so far not much except for LRUBlock cache spitting > >>> metrics .. > >>> > >>> 2011-04-20 12:28:48,375 DEBUG > >>> org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=8.26 > >>> MB, free=190.08 MB, max=198.34 MB, blocks=112, accesses=55732209, > >>> hits=55732083, hitRatio=99.99%%, cachingAccesses=55732195, > >>> cachingHits=55732083, cachingHitsRatio=99.99%%, evictions=0, > >>> evicted=0, evictedPerRun=NaN > >>> 2011-04-20 12:33:48,375 DEBUG > >>> org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=8.26 > >>> MB, free=190.08 MB, max=198.34 MB, blocks=112, accesses=56703200, > >>> hits=56703074, hitRatio=99.99%%, cachingAccesses=56703186, > >>> cachingHits=56703074, cachingHitsRatio=99.99%%, evictions=0, > >>> evicted=0, evictedPerRun=NaN > >>> 2011-04-20 12:38:48,375 DEBUG > >>> org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=8.26 > >>> MB, free=190.08 MB, max=198.34 MB, blocks=112, accesses=57708231, > >>> hits=57708105, hitRatio=99.99%%, cachingAccesses=57708217, > >>> cachingHits=57708105, cachingHitsRatio=99.99%%, evictions=0, > >>> evicted=0, evictedPerRun=NaN > >>> > >>> > >>> On Wed, Apr 20, 2011 at 12:35 PM, Stack <[email protected]> wrote: > >>>> If one region only, then its located on a single regionserver. Tail > >>>> that regionservers logs. It might tell us something. > >>>> St.Ack > >>>> > >>>> On Wed, Apr 20, 2011 at 12:25 PM, Stack <[email protected]> wrote: > >>>>> On Wed, Apr 20, 2011 at 12:25 PM, Stack <[email protected]> wrote: > >>>>>> On Tue, Apr 19, 2011 at 4:46 PM, Dmitriy Lyubimov < > [email protected]> wrote: > >>>>>>> Right now i am shooting scans returning between 3 and 40 rows and > >>>>>>> regardless of data size, approximately 500-400 QPS. The data tables > >>>>>>> are almost empty and in-memory, so they surely should fit in those > 40% > >>>>>>> heap dedicated to them. > >>>>>>> > >>>>>> > >>>>>> How many clients are going against the cluster? If you use less, do > >>>>>> your numbers improve? > >>>>>> > >>>>> > >>>>> And all these clients are going against a single 40 row table? > >>>>> St.Ack > >>>>> > >>>> > >>> > >> > > >
