So i can't seem to be able to immediately find the explanation for those metrics
- rpcQueueTime -- do I assume it correctly it's the time a request sits waiting int the incoming rpc queue before being picked up by handler ? -rpcProcessingTime -- do i assume it correctly it's time of request being processed by region server's handler? So inner time to last byte should be approximately sum of those, right? Thanks. -Dmitriy On Wed, Apr 20, 2011 at 1:17 PM, Dmitriy Lyubimov <[email protected]> wrote: > Yes that's what i said. there's metric for fs latency but we are not > hitting it so it's not useful. > > Question is which one might be useful to measure inner ttlb, and i > don't see it there. > > On Wed, Apr 20, 2011 at 1:14 PM, Ted Dunning <[email protected]> wrote: >> FS latency shouldn't matter with your 99.9% cache hit rate as reported. >> >> On Wed, Apr 20, 2011 at 12:55 PM, Dmitriy Lyubimov <[email protected]>wrote: >> >>> Yes -- I already looked thru 'regionserver' metrics some time ago in >>> hbase book. And i am not sure there's a 'inner ttlb' metric. >>> >>> There are fs latency metrics there but nothing for the respons times. >>> fs latency is essentially hdfs latency AFAICT and that would not be >>> relevant to what i am asking for (for as long as we are hitting LRU >>> block cache anyway). we are not hitting fs. >>> >>> Unless there are more metrics than listed in the Hbase Book? >>> >>> >>> On Wed, Apr 20, 2011 at 12:46 PM, Stack <[email protected]> wrote: >>> > Enable rpc logging. Will show in your ganglia. See metrics article >>> > on hbase home page. >>> > >>> > On Wed, Apr 20, 2011 at 12:44 PM, Dmitriy Lyubimov <[email protected]> >>> wrote: >>> >> Is there any way to log 'inner' TTLB times the region server incurs for >>> reads? >>> >> >>> >> >>> >> On Wed, Apr 20, 2011 at 12:43 PM, Dmitriy Lyubimov <[email protected]> >>> wrote: >>> >>> i just enabled debug logging for o.a.h.hbase logger in that particular >>> >>> region server... so far not much except for LRUBlock cache spitting >>> >>> metrics .. >>> >>> >>> >>> 2011-04-20 12:28:48,375 DEBUG >>> >>> org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=8.26 >>> >>> MB, free=190.08 MB, max=198.34 MB, blocks=112, accesses=55732209, >>> >>> hits=55732083, hitRatio=99.99%%, cachingAccesses=55732195, >>> >>> cachingHits=55732083, cachingHitsRatio=99.99%%, evictions=0, >>> >>> evicted=0, evictedPerRun=NaN >>> >>> 2011-04-20 12:33:48,375 DEBUG >>> >>> org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=8.26 >>> >>> MB, free=190.08 MB, max=198.34 MB, blocks=112, accesses=56703200, >>> >>> hits=56703074, hitRatio=99.99%%, cachingAccesses=56703186, >>> >>> cachingHits=56703074, cachingHitsRatio=99.99%%, evictions=0, >>> >>> evicted=0, evictedPerRun=NaN >>> >>> 2011-04-20 12:38:48,375 DEBUG >>> >>> org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=8.26 >>> >>> MB, free=190.08 MB, max=198.34 MB, blocks=112, accesses=57708231, >>> >>> hits=57708105, hitRatio=99.99%%, cachingAccesses=57708217, >>> >>> cachingHits=57708105, cachingHitsRatio=99.99%%, evictions=0, >>> >>> evicted=0, evictedPerRun=NaN >>> >>> >>> >>> >>> >>> On Wed, Apr 20, 2011 at 12:35 PM, Stack <[email protected]> wrote: >>> >>>> If one region only, then its located on a single regionserver. Tail >>> >>>> that regionservers logs. It might tell us something. >>> >>>> St.Ack >>> >>>> >>> >>>> On Wed, Apr 20, 2011 at 12:25 PM, Stack <[email protected]> wrote: >>> >>>>> On Wed, Apr 20, 2011 at 12:25 PM, Stack <[email protected]> wrote: >>> >>>>>> On Tue, Apr 19, 2011 at 4:46 PM, Dmitriy Lyubimov < >>> [email protected]> wrote: >>> >>>>>>> Right now i am shooting scans returning between 3 and 40 rows and >>> >>>>>>> regardless of data size, approximately 500-400 QPS. The data tables >>> >>>>>>> are almost empty and in-memory, so they surely should fit in those >>> 40% >>> >>>>>>> heap dedicated to them. >>> >>>>>>> >>> >>>>>> >>> >>>>>> How many clients are going against the cluster? If you use less, do >>> >>>>>> your numbers improve? >>> >>>>>> >>> >>>>> >>> >>>>> And all these clients are going against a single 40 row table? >>> >>>>> St.Ack >>> >>>>> >>> >>>> >>> >>> >>> >> >>> > >>> >> >
