Some of our servers have 5.2gb hbase heaps with the standard 40% (2gb) memstore and 20% (1gb) blockcache. I'm wondering where the indexes and bloom filters get counted. Is it in that remaining 2gb, and are the bloom filters counted in the storefileIndexSize=566 metric?
The overall data in each server isn't too large on disk (~50gb) after gzip compression of 25x, but it's made of long keys with short values so there is a lot of metadata. I'm thinking of upping the block size to 256k but thought i'd ask how it worked first. some metrics: regions=401, stores=401, storefiles=758, storefileIndexSize=566, memstoreSize=1793, usedHeap=3658, maxHeap=5183, blockCacheSize=816306600 Thanks for the help, Matt On Fri, Apr 15, 2011 at 10:39 AM, Doug Meil <[email protected]>wrote: > The HBase book has come through some updates recently on these metrics. > > http://hbase.apache.org/book.html#hbase_metrics > > > -----Original Message----- > From: [email protected] [mailto:[email protected]] On Behalf Of Stack > Sent: Friday, April 15, 2011 5:04 AM > To: [email protected] > Cc: 陈加俊; [email protected] > Subject: Re: heap memory allocation > > On Thu, Apr 14, 2011 at 9:50 PM, 陈加俊 <[email protected]> wrote: > > How does heap memory allocated in regionserver ? > > > > Metrics: > > > > request=0.0, regions=1283, stores=2304, storefiles=1968, > > storefileIndexSize=246, memstoreSize=791, compactionQueueSize=0, > > usedHeap=5669, maxHeap=11991, blockCacheSize=2095157424, > > blockCacheFree=419681872, blockCacheCount=24410, > > blockCacheHitRatio=92, fsReadLatency=0, fsWriteLatency=0, > > fsSyncLatency=0 > > > > > > > > maxHeap=11991 > > usedHeap=5669 > > > > This just the state of the JVM memory bean at the last update time. > Looks like you have given hbase 12G. We're using about 5.5 Gig. See > here for where we read it from the bean: > > http://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/HRegionServer.html#751 > These look like they report in MB. > > > blockCacheSize=2095157424 > > memstoreSize=791 > > ???? > > These look like they report in bytes and MB. Should make them all same > units. HBase has a cache of hdfs blocks. And memstore is where we write > edits too before they get flushed to the filesystem. > > > > > Free=maxHeap-usedHeap? > > > > Read the javadoc on the bean returned by > ManagementFactory.getMemoryMXBean() call. > > St.Ack > > > > > -- > > Thanks & Best regards > > jiajun > > >
