Actually, having a smaller heap will decrease the risk of a catastrophic GC. It probably wil also increase the likelihood of a full GC.
Having a larger heap will let you go long without a full GC, but with a very large heap a full GC may take your region server off-line long enough to be considered a failure. Then you will have cascading badness. On Fri, Feb 18, 2011 at 12:05 PM, Chris Tarnas <[email protected]> wrote: > Thank you , ad that bring me to my next question... > > What is the current recommendation on the max heap size for Hbase if RAM on > the server is not an issue? Right now I am at 8GB and have no issues, can I > safely do 12GB? The servers have plenty of RAM (48GB) so that should not be > an issue - I just want to minimize the risk that GC will cause problems. > > thanks again. > -chris > > On Feb 18, 2011, at 11:59 AM, Jean-Daniel Cryans wrote: > > > That's what I usually recommend, the bigger the flushed files the > > better. On the other hand, you only have so much memory to dedicate to > > the MemStore... > > > > J-D > > > > On Fri, Feb 18, 2011 at 11:50 AM, Chris Tarnas <[email protected]> wrote: > >> Would it be a good idea to raise the hbase.hregion.memstore.flush.size > if you have really large regions? > >> > >> -chris > >> > >> On Feb 18, 2011, at 11:43 AM, Jean-Daniel Cryans wrote: > >> > >>> Less regions, but it's often a good thing if you have a lot of data :) > >>> > >>> It's probably a good thing to bump the HDFS block size to 128 or 256MB > >>> since you know you're going to have huge-ish files. > >>> > >>> But anyway regarding penalties, I can't think of one that clearly > >>> comes out (unless you use a very small heap). The IO usage patterns > >>> will change, but unless you flush very small files all the time and > >>> need to recompact them into much bigger ones, then it shouldn't really > >>> be an issue. > >>> > >>> J-D > >>> > >>> On Fri, Feb 18, 2011 at 11:36 AM, Jason Rutherglen > >>> <[email protected]> wrote: > >>>>> We are also using a 5Gb region size to keep our region > >>>>> counts in the 100-200 range/node per Jonathan Grey's recommendation. > >>>> > >>>> So there isn't a penalty incurred from increasing the max region size > >>>> from 256MB to 5GB? > >>>> > >> > >> > >
