HI, what i mean is shoud we increate the memstore according to this log?
Thanks! beatls. On Thu, Jan 3, 2013 at 8:37 PM, Anoop Sam John <[email protected]> wrote: > > I guess you mean "what is blocking memstore size" > This you can configure using 2 properties. > 1. hbase.hregion.memstore.flush.size -> Using this you specify when the > memstore to be flushed as a file. The default value is 128MB. So when the > region memstore size reaches this value a flush will be initiated. > 2. hbase.hregion.memstore.block.multiplier -> deafult value is 2 > The blocking memstore size is calculated as flushSize multiplied by this > multiplier. So the default value will be 256MB. > What value to set depends on the total memory available for the process in > the RegionServers and the total no# of regions per server. > Say you set this blocking size as 512m and there can be 100 regions per > region server in your cluster, then you are *possibly allowing* a total > memory of 512*100 mb for the memstores alone. [RS process will not other > memory also for its normal operation]. Take care of these aspects when you > design > > -Anoop- > > ________________________________________ > From: hua beatls [[email protected]] > Sent: Thursday, January 03, 2013 5:41 PM > To: [email protected] > Subject: Re: reginoserver's log analysis? > > HI, > what is the block size? how to set it ? and shoud i increate it for > the memstore ? > > Thanks! > beatls > > On Thu, Jan 3, 2013 at 6:23 PM, Anoop Sam John <[email protected]> wrote: > > > The writes to your table will be written 1st to in memory data structure > > (memstore). When this memstore reaches some defined size it will be > flushed > > into the file system and the memory can be cleared. > > > > The flush operation may take some time as IO write is involved, During > > this time also the client may be writing data to the table. Well HBase > > allows to do so and the data there in memory for a region can be becoming > > more and more. So there will be a max amount of total data which can be > > there in memory. If that is crossed HBase will stop taking any writes > from > > the client until the flush is over and the memory got cleared. As per > the > > log in your cluster the blocking memstore size is 256m. > > > > Hope you got the idea. > > > > -Anoop- > > ________________________________________ > > From: hua beatls [[email protected]] > > Sent: Thursday, January 03, 2013 3:15 PM > > To: [email protected] > > Subject: reginoserver's log analysis? > > > > Hi, > > below is the regionserver's log, what does the red marked "blocking > > updates" mean ? > > any indication from " memstore size 256.8m is >= than blocking 256.0m > > size" > > > > > > > > 42,651 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking > > updatesfor 'IPC Server handler 1 on 60020' on region > > > > > BT_NET_LOG_999,0000189020000000000000000000000000000000,1357201912899.efd6fd4581483aded26175300c48b7b7.: > > memstore size 256.8m is >= than blocking 256.0m size > > > > 2013-01-03 17:00:42,717 INFO > > org.apache.hadoop.hbase.regionserver.HRegion: Blocking > > updates for 'IPC Server handler 27 on 60020' on region > > > > > BT_NET_LOG_999,0000189020000000000000000000000000000000,1357201912899.efd6fd4581483aded26175300c48b7b7.: > > memstore size 256.8m is >= than blocking 256.0m size > > > > 2013-01-03 17:00:42,761 INFO > org.apache.hadoop.hbase.regionserver.HRegion: > > Blocking updates for 'IPC Server handler 50 on 60020' on region > > > > > BT_NET_LOG_999,0000189020000000000000000000000000000000,1357201912899.efd6fd4581483aded26175300c48b7b7.: > > memstore size 256.8m is >= than blocking 256.0m size > > > > 2013-01-03 17:00:42,875 INFO > org.apache.hadoop.hbase.regionserver.HRegion: > > Blocking updates for 'IPC Server handler 55 on 60020' on region > > > > > BT_NET_LOG_999,0000189020000000000000000000000000000000,1357201912899.efd6fd4581483aded26175300c48b7b7.: > > memstore size 256.8m is >= than blocking 256.0m size > > > > > > Thanks! > > > > beatls > > >
