If using Java 7 and G1, you might want to look over: https://software.intel.com/en-us/blogs/2014/06/18/part-1-tuning-java-garbage-collection-for-hbase
On Wed, Aug 20, 2014 at 8:26 AM, Jean-Marc Spaggiari < [email protected]> wrote: > I agree with Bryan. > > HBase start to have some GC difficulties after 16GB. Depending of the kind > of load you put on it, it might be fine up to a certain point. Seems that > with your daily load, 30GB is fine. However, when you start to do the > compactions, you start to see the GC issues. > > You can try G1. There will be a performance impact, but might get you on a > more stable path. Else, you will have to reduce your heap size, but this, > again, might have some impact on the performances too... Basically, what > ever you will do to get more stable will impact your performances :( > > JM > > > 2014-08-20 6:10 GMT-04:00 Bryan Beaudreault <[email protected]>: > > > You're gonna want to use java7 and the G1 collector with a heap that > high. > > I'm surprised you aren't seeing other issues. > > > > It's possible that with the added CPU load from the compactions, the > > garbage collector is not able to keep up and must do a full clean. > > > > On Wednesday, August 20, 2014, yanivG <[email protected]> wrote: > > > > > Yes. > > > The RegionServer cmd arguments are: > > > /usr/java/latest//bin/java -XX:OnOutOfMemoryError=kill -9 %p -Xmx30720m > > > -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+UseNUMA > > > -XX:CMSInitiatingOccupancyFraction=60 -ea > -XX:+HeapDumpOnOutOfMemoryError > > > -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+UseNUMA > > > -XX:CMSInitiatingOccupancyFraction=60 -ea > -XX:+HeapDumpOnOutOfMemoryError > > > > > > > > > free -g output: > > > total used free shared > > > buffers cached > > > Mem: 126 114 11 0 > > 0 > > > 90 > > > -/+ buffers/cache: 23 102 > > > Swap: 9 0 9 > > > > > > meminfo > > > > > > MemTotal: 132145596 kB > > > MemFree: 11875380 kB > > > Buffers: 359804 kB > > > Cached: 94909280 kB > > > SwapCached: 0 kB > > > Active: 69301468 kB > > > Inactive: 47497184 kB > > > Active(anon): 18676840 kB > > > Inactive(anon): 2858304 kB > > > Active(file): 50624628 kB > > > Inactive(file): 44638880 kB > > > > > > > > > > > > > > > > > > > > > > > > -- > > > View this message in context: > > > > > > http://apache-hbase.679495.n3.nabble.com/GC-peaks-during-major-compaction-tp4062704p4062930.html > > > Sent from the HBase User mailing list archive at Nabble.com. > > > > > > -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
