So you use Bulk load with HFileOpFormat for writing data? Then you can reduce the hbase.regionserver.global.memstore.upperLimit and hbase.regionserver.global.memstore.lowerLimit and give more heap % for the block cache. Not getting why u try to reduce that also.
-Anoop- On Thu, Nov 28, 2013 at 3:16 PM, Ivan Tretyakov <[email protected] > wrote: > Hi! > > We are using HBase 0.92.1-cdh4.1.1. To import data the only way we use is > bulk load. And our common access pattern is sequential scans of different > parts of the tables. > > Since that we are considering to disable block cache by setting > hbase.block.cache.size to zero. > But We've found following in HBase book ( > http://hbase.apache.org/book/important_configurations.html): > > "Do not turn off block cache (You'd do it by setting hbase.block.cache.size > to zero). Currently we do not do well if you do this because the > regionserver will spend all its time loading hfile indices over and over > again. If your working set it such that block cache does you no good, at > least size the block cache such that hfile indices will stay up in the > cache (you can get a rough idea on the size you need by surveying > regionserver UIs; you'll see index block size accounted near the top of the > webpage)." > > Another thing we consider to reduce is memstore size by tuning following > options hbase.regionserver.global.memstore.upperLimit and > hbase.regionserver.global.memstore.lowerLimit. > > So, my questions are: > > Does it make sense to touch these options in our case? > Is this memory reserved or other processes inside regionserver can use it? > > Thanks in advance! > > -- > Best Regards > Ivan Tretyakov > > Deployment Engineer > Grid Dynamics > +7 812 640 38 > 76<https://mail.google.com/mail/u/0/html/compose/static_files/blank_quirks.html#> > Skype: ivan.v.tretyakov > www.griddynamics.com > [email protected] >
