Swapping and Java simply don't go well together. You need to ensure that the committed memory is smaller than the available memory. Also see http://hbase.apache.org/book.html#perf.os.swap
I haven't looked closely at your GC output but even if CMS was kicking as early as it's supposed to, the fact that you are swapping might just screw up everything. J-D On Mon, Oct 7, 2013 at 3:13 PM, prakash kadel <[email protected]>wrote: > BTW, > if i disable the swap at all. What will happen in the above situation? > currently it starts swapping at 90% > > Sincerely > > > On Tue, Oct 8, 2013 at 7:09 AM, prakash kadel <[email protected] > >wrote: > > > thanks, > > > > yup, it seems so. I have 48 gb memory. i see it swaps at that point. > > > > btw, why is the CMS not kicking in early? do you have any idea? > > > > sincerely > > > > > > > > On Tue, Oct 8, 2013 at 3:00 AM, Jean-Daniel Cryans <[email protected] > >wrote: > > > >> This line: > >> > >> [CMS-concurrent-mark: 12.929/88.767 secs] [Times: user=14.30 sys=3.74, > >> real=88.77 > >> secs] > >> > >> Is suspicious. Are you swapping? > >> > >> J-D > >> > >> > >> On Mon, Oct 7, 2013 at 8:34 AM, prakash kadel <[email protected] > >> >wrote: > >> > >> > Also, > >> > why is the CMS not kicking in early, i have set XX:+ > >> > UseCMSInitiatingOccupancyOnly??? > >> > > >> > Sincerely, > >> > Prakash > >> > > >> > > >> > On Tue, Oct 8, 2013 at 12:32 AM, prakash kadel < > [email protected] > >> > >wrote: > >> > > >> > > Hello, > >> > > I am getting this YADE all the time > >> > > > >> > > HBASE_HEAPSIZE=8000 > >> > > > >> > > Settings: -ea -XX:+UseConcMarkSweepGC -XX:MaxGCPauseMillis=200 > >> > > -XX:+HeapDumpOnOutOfMemoryError -XX:+CMSIncrementalMode > >> -XX:+UseParNewGC > >> > > -XX:CMSInitiatingOccupancyFraction=50 > >> -XX:+UseCMSInitiatingOccupancyOnly > >> > > -XX:NewSize=256m -XX:MaxNewSize=256m > >> > > > >> > > it seems there is promotion failure and the CMS take too long > >> > > > >> > > 2013-10-07T01:22:55.784+0900: [GC [ParNew: 235968K->26176K(235968K), > >> > > 0.3219980 secs] 7709485K->7538063K(8165824K) icms_dc=0 , 0.3221100 > >> secs] > >> > > [Times: user=0.27 sys=0.01, real=0.33 secs] > >> > > 2013-10-07T01:23:07.361+0900: [GC [ParNew: 235842K->26176K(235968K), > >> > > 0.1899680 secs] 7747729K->7578713K(8165824K) icms_dc=0 , 0.1900700 > >> secs] > >> > > [Times: user=0.26 sys=0.02, real=0.19 secs] > >> > > 2013-10-07T01:23:20.154+0900: [GC [ParNew: 235803K->26176K(235968K), > >> > > 0.2428200 secs] 7788341K->7615284K(8165824K) icms_dc=0 , 0.2429570 > >> secs] > >> > > [Times: user=0.25 sys=0.02, real=0.24 secs] > >> > > 2013-10-07T01:23:34.594+0900: [GC [ParNew: 235889K->26176K(235968K), > >> > > 0.2440980 secs] 7824998K->7651179K(8165824K) icms_dc=0 , 0.2442130 > >> secs] > >> > > [Times: user=0.20 sys=0.03, real=0.25 secs] > >> > > 2013-10-07T01:23:47.666+0900: [GC [ParNew: 235906K->26176K(235968K), > >> > > 0.2998100 secs] 7860909K->7686832K(8165824K) icms_dc=3 , 0.3020280 > >> secs] > >> > > [Times: user=0.23 sys=0.04, real=0.30 secs] > >> > > 2013-10-07T01:23:57.216+0900: [GC [1 CMS-initial-mark: > >> > 7660656K(7929856K)] > >> > > 7788778K(8165824K), 3.7665320 secs] [Times: user=0.07 sys=0.06, > >> real=3.77 > >> > > secs] > >> > > 2013-10-07T01:24:05.508+0900: [GC [ParNew: 235811K->26176K(235968K), > >> > > 0.4632860 secs] 7896468K->7721167K(8165824K) icms_dc=3 , 0.4634100 > >> secs] > >> > > [Times: user=0.21 sys=0.03, real=0.46 secs] > >> > > 2013-10-07T01:24:19.889+0900: [GC [ParNew: 235812K->26176K(235968K), > >> > > 0.3531980 secs] 7930804K->7755633K(8165824K) icms_dc=3 , 0.3533230 > >> secs] > >> > > [Times: user=0.24 sys=0.06, real=0.35 secs] > >> > > 2013-10-07T01:24:32.832+0900: [GC [ParNew: 235968K->26176K(235968K), > >> > > 0.6298370 secs] 7965425K->7790643K(8165824K) icms_dc=3 , 0.6299530 > >> secs] > >> > > [Times: user=0.23 sys=0.03, real=0.63 secs] > >> > > 2013-10-07T01:24:43.629+0900: [GC [ParNew: 235800K->26176K(235968K), > >> > > 0.3190580 secs] 8000268K->7825555K(8165824K) icms_dc=3 , 0.3191840 > >> secs] > >> > > [Times: user=0.24 sys=0.02, real=0.32 secs] > >> > > 2013-10-07T01:24:56.005+0900: [GC [ParNew: 235848K->26176K(235968K), > >> > > 0.4839400 secs] 8035228K->7860300K(8165824K) icms_dc=3 , 0.4840480 > >> secs] > >> > > [Times: user=0.31 sys=0.03, real=0.49 secs] > >> > > 2013-10-07T01:25:07.282+0900: [GC [ParNew: 235750K->26176K(235968K), > >> > > 0.3423250 secs] 8069875K->7895852K(8165824K) icms_dc=9 , 0.3424380 > >> secs] > >> > > [Times: user=0.21 sys=0.06, real=0.34 secs] > >> > > 2013-10-07T01:25:19.853+0900: [GC [ParNew (promotion failed): > >> > > 235745K->235745K(235968K), 0.3339710 > >> > secs][CMS2013-10-07T01:25:29.750+0900: > >> > > [CMS-concurrent-mark: 12.929/88.767 secs] [Times: user=14.30 > sys=3.74, > >> > > real=88.77 secs] > >> > > (concurrent mode failure): 7899125K->2882954K(7929856K), 42.8279810 > >> > secs] > >> > > 8105422K->2882954K(8165824K), [CMS Perm : 31956K->31861K(53340K)] > >> > icms_dc=9 > >> > > , 43.1621090 secs] [Times: user=10.40 sys=1.89, real=43.16 secs] > >> > > 2013-10-07T01:26:08.288+0900: [GC [1 CMS-initial-mark: > >> > 2882954K(7929856K)] > >> > > 2978434K(8165824K), 0.0965830 secs] [Times: user=0.04 sys=0.00, > >> real=0.09 > >> > > secs] > >> > > Heap > >> > > par new generation total 235968K, used 197697K > [0x0000000606e00000, > >> > > 0x0000000616e00000, 0x0000000616e00000) > >> > > eden space 209792K, 94% used [0x0000000606e00000, > >> 0x0000000612f10718, > >> > > 0x0000000613ae0000) > >> > > from space 26176K, 0% used [0x0000000615470000, > >> 0x0000000615470000, > >> > > 0x0000000616e00000) > >> > > to space 26176K, 0% used [0x0000000613ae0000, > >> 0x0000000613ae0000, > >> > > 0x0000000615470000) > >> > > concurrent mark-sweep generation total 7929856K, used 2882954K > >> > > [0x0000000616e00000, 0x00000007fae00000, 0x00000007fae00000) > >> > > concurrent-mark-sweep perm gen total 53340K, used 31960K > >> > > [0x00000007fae00000, 0x00000007fe217000, 0x0000000800000000) > >> > > > >> > > What is wrong here? please give me some suggestions. > >> > > > >> > > > >> > > Sincerely, > >> > > Prakash > >> > > > >> > > >> > > > > >
