The bigger the heap the longer the GC pause of the world when fragmentation requires it, 8GB is "safer".
In 0.90.1 you can try enabling the new memstore allocator that seems to do a really good job, checkout the jira first: https://issues.apache.org/jira/browse/HBASE-3455 J-D On Fri, Feb 18, 2011 at 12:05 PM, Chris Tarnas <[email protected]> wrote: > Thank you , ad that bring me to my next question... > > What is the current recommendation on the max heap size for Hbase if RAM on > the server is not an issue? Right now I am at 8GB and have no issues, can I > safely do 12GB? The servers have plenty of RAM (48GB) so that should not be > an issue - I just want to minimize the risk that GC will cause problems. > > thanks again. > -chris > > On Feb 18, 2011, at 11:59 AM, Jean-Daniel Cryans wrote: > >> That's what I usually recommend, the bigger the flushed files the >> better. On the other hand, you only have so much memory to dedicate to >> the MemStore... >> >> J-D >> >> On Fri, Feb 18, 2011 at 11:50 AM, Chris Tarnas <[email protected]> wrote: >>> Would it be a good idea to raise the hbase.hregion.memstore.flush.size if >>> you have really large regions? >>> >>> -chris >>> >>> On Feb 18, 2011, at 11:43 AM, Jean-Daniel Cryans wrote: >>> >>>> Less regions, but it's often a good thing if you have a lot of data :) >>>> >>>> It's probably a good thing to bump the HDFS block size to 128 or 256MB >>>> since you know you're going to have huge-ish files. >>>> >>>> But anyway regarding penalties, I can't think of one that clearly >>>> comes out (unless you use a very small heap). The IO usage patterns >>>> will change, but unless you flush very small files all the time and >>>> need to recompact them into much bigger ones, then it shouldn't really >>>> be an issue. >>>> >>>> J-D >>>> >>>> On Fri, Feb 18, 2011 at 11:36 AM, Jason Rutherglen >>>> <[email protected]> wrote: >>>>>> We are also using a 5Gb region size to keep our region >>>>>> counts in the 100-200 range/node per Jonathan Grey's recommendation. >>>>> >>>>> So there isn't a penalty incurred from increasing the max region size >>>>> from 256MB to 5GB? >>>>> >>> >>> > >
