Re: You Are Dead Exception due to promotion failure

2013-11-01 Thread Jean-Daniel Cryans 
It reads that it spent 89 seconds doing a CMS concurrent mark, but really
just spent 14 seconds of user CPU and 4 seconds of system CPU doing it.
Where are the other 70 seconds? It's often just swapping, and less likely
it can also be CPU starvation.

J-D


On Fri, Nov 1, 2013 at 1:40 AM, Asaf Mesika asaf.mes...@gmail.com wrote:

 Can you please explain why is this suspicious?

 On Monday, October 7, 2013, Jean-Daniel Cryans wrote:

  This line:
 
  [CMS-concurrent-mark: 12.929/88.767 secs] [Times: user=14.30 sys=3.74,
  real=88.77
  secs]
 
  Is suspicious. Are you swapping?
 
  J-D
 
 
  On Mon, Oct 7, 2013 at 8:34 AM, prakash kadel prakash.ka...@gmail.com
 javascript:;
  wrote:
 
   Also,
  why is the CMS not kicking in early, i have set XX:+
   UseCMSInitiatingOccupancyOnly???
  
   Sincerely,
   Prakash
  
  
   On Tue, Oct 8, 2013 at 12:32 AM, prakash kadel 
 prakash.ka...@gmail.com
   wrote:
  
Hello,
  I am getting this YADE all the time
   
HBASE_HEAPSIZE=8000
   
Settings: -ea -XX:+UseConcMarkSweepGC -XX:MaxGCPauseMillis=200
-XX:+HeapDumpOnOutOfMemoryError -XX:+CMSIncrementalMode
  -XX:+UseParNewGC
-XX:CMSInitiatingOccupancyFraction=50
  -XX:+UseCMSInitiatingOccupancyOnly
-XX:NewSize=256m -XX:MaxNewSize=256m
   
it seems there is promotion failure and the CMS take too long
   
2013-10-07T01:22:55.784+0900: [GC [ParNew: 235968K-26176K(235968K),
0.3219980 secs] 7709485K-7538063K(8165824K) icms_dc=0 , 0.3221100
  secs]
[Times: user=0.27 sys=0.01, real=0.33 secs]
2013-10-07T01:23:07.361+0900: [GC [ParNew: 235842K-26176K(235968K),
0.1899680 secs] 7747729K-7578713K(8165824K) icms_dc=0 , 0.1900700
  secs]
[Times: user=0.26 sys=0.02, real=0.19 secs]
2013-10-07T01:23:20.154+0900: [GC [ParNew: 235803K-26176K(235968K),
0.2428200 secs] 7788341K-7615284K(8165824K) icms_dc=0 , 0.2429570
  secs]
[Times: user=0.25 sys=0.02, real=0.24 secs]
2013-10-07T01:23:34.594+0900: [GC [ParNew: 235889K-26176K(235968K),
0.2440980 secs] 7824998K-7651179K(8165824K) icms_dc=0 , 0.2442130
  secs]
[Times: user=0.20 sys=0.03, real=0.25 secs]
2013-10-07T01:23:47.666+0900: [GC [ParNew: 235906K-26176K(235968K),
0.2998100 secs] 7860909K-7686832K(8165824K) icms_dc=3 , 0.3020280
  secs]
[Times: user=0.23 sys=0.04, real=0.30 secs]
2013-10-07T01:23:57.216+0900: [GC [1 CMS-initial-mark:
   7660656K(7929856K)]
7788778K(8165824K), 3.7665320 secs] [Times: user=0.07 sys=0.06,
  real=3.77
secs]
2013-10-07T01:24:05.508+0900: [GC [ParNew: 235811K-26176K(235968K),
0.4632860 secs] 7896468K-7721167K(8165824K) icms_dc=3 , 0.4634100
  secs]
[Times: user=0.21 sys=0.03, real=0.46 secs]
2013-10-07T01:24:19.889+0900: [GC [ParNew: 235812K-26176K(235968K),
0.3531980 secs] 7930804K-7755633K(8165824K) icms_dc=3 , 0.3533230
  secs]
[Times: user=0.24 sys=0.06, real=0.35 secs]
2013-10-07T01:24:32.832+0900: [GC [ParNew: 235968K-26176K(235968K),
0.6298370 secs] 7965425K-7790643K(8165824K) icms_dc=3 , 0.6299530
  secs]
[Times: user=0.23 sys=0.03, real=0.63 secs]
2013-10-07T01:24:43.629+0900: [GC [ParNew: 235800K-26176K(235968K),
0.3190580 secs] 8000268K-782K(8165824K) icms_dc=3 , 0.3191840
  secs]
[Times: user=0.24 sys=0.02, real=0.32 secs]
2013-10-07T01:24:56.005+0900: [GC [ParNew: 235848K-26176K(235968K),
0.4839400 secs] 8035228K-7860300K(8165824K) icms_dc=3 , 0.4840480
  secs]
[Times: user=0.31 sys=0.03, real=0.49 secs]
2013-10-07T01:25:07.282+0900: [GC [ParNew: 235750K-26176K(235968K),
0.3423250 secs] 8069875K-7895852K(8165824K) icms_dc=9 , 0.3424380
  secs]
[Times: user=0.21 sys=0.06, real=0.34 secs]
2013-10-07T01:25:19.853+0900: [GC [ParNew (promotion failed):
235745K-235745K(235968K), 0.3339710
   secs][CMS2013-10-07T01:25:29.750+0900:
[CMS-concurrent-mark: 12.929/88.767 secs] [Times: user=14.30
 sys=3.74,
real=88.77 secs]
 (concurrent mode failure): 7899125K-2882954K(7929856K), 42.8279810
   secs]
8105422K-2882954K(8165824K), [CMS Perm : 31956K-31861K(53340K)]
   icms_dc=9
, 43.1621090 secs] [Times: user=10.40 sys=1.89, real=43.16 secs]
2013-10-07T01:26:08.288+0900: [GC [1 CMS-initial-mark:
   2882954K(7929856K)]
2978434K(8165824K), 0.0965830 secs] [Times: user=0.04 sys=0.00,
  real=0.09
secs]
Heap
 par new generation   total 235968K, used 197697K
 [0x000606e0,
0x000616e0, 0x000616e0)
  eden space 209792K,  94% used [0x000606e0,
  0x000612f10718,
0x000613ae)
  from space 26176K,   0% used [0x00061547,
 0x00061547,

Re: You Are Dead Exception due to promotion failure

2013-10-07 Thread prakash kadel 
Also,
   why is the CMS not kicking in early, i have set XX:+
UseCMSInitiatingOccupancyOnly???

Sincerely,
Prakash


On Tue, Oct 8, 2013 at 12:32 AM, prakash kadel prakash.ka...@gmail.comwrote:

 Hello,
   I am getting this YADE all the time

 HBASE_HEAPSIZE=8000

 Settings: -ea -XX:+UseConcMarkSweepGC -XX:MaxGCPauseMillis=200
 -XX:+HeapDumpOnOutOfMemoryError -XX:+CMSIncrementalMode -XX:+UseParNewGC
 -XX:CMSInitiatingOccupancyFraction=50 -XX:+UseCMSInitiatingOccupancyOnly
 -XX:NewSize=256m -XX:MaxNewSize=256m

 it seems there is promotion failure and the CMS take too long

 2013-10-07T01:22:55.784+0900: [GC [ParNew: 235968K-26176K(235968K),
 0.3219980 secs] 7709485K-7538063K(8165824K) icms_dc=0 , 0.3221100 secs]
 [Times: user=0.27 sys=0.01, real=0.33 secs]
 2013-10-07T01:23:07.361+0900: [GC [ParNew: 235842K-26176K(235968K),
 0.1899680 secs] 7747729K-7578713K(8165824K) icms_dc=0 , 0.1900700 secs]
 [Times: user=0.26 sys=0.02, real=0.19 secs]
 2013-10-07T01:23:20.154+0900: [GC [ParNew: 235803K-26176K(235968K),
 0.2428200 secs] 7788341K-7615284K(8165824K) icms_dc=0 , 0.2429570 secs]
 [Times: user=0.25 sys=0.02, real=0.24 secs]
 2013-10-07T01:23:34.594+0900: [GC [ParNew: 235889K-26176K(235968K),
 0.2440980 secs] 7824998K-7651179K(8165824K) icms_dc=0 , 0.2442130 secs]
 [Times: user=0.20 sys=0.03, real=0.25 secs]
 2013-10-07T01:23:47.666+0900: [GC [ParNew: 235906K-26176K(235968K),
 0.2998100 secs] 7860909K-7686832K(8165824K) icms_dc=3 , 0.3020280 secs]
 [Times: user=0.23 sys=0.04, real=0.30 secs]
 2013-10-07T01:23:57.216+0900: [GC [1 CMS-initial-mark: 7660656K(7929856K)]
 7788778K(8165824K), 3.7665320 secs] [Times: user=0.07 sys=0.06, real=3.77
 secs]
 2013-10-07T01:24:05.508+0900: [GC [ParNew: 235811K-26176K(235968K),
 0.4632860 secs] 7896468K-7721167K(8165824K) icms_dc=3 , 0.4634100 secs]
 [Times: user=0.21 sys=0.03, real=0.46 secs]
 2013-10-07T01:24:19.889+0900: [GC [ParNew: 235812K-26176K(235968K),
 0.3531980 secs] 7930804K-7755633K(8165824K) icms_dc=3 , 0.3533230 secs]
 [Times: user=0.24 sys=0.06, real=0.35 secs]
 2013-10-07T01:24:32.832+0900: [GC [ParNew: 235968K-26176K(235968K),
 0.6298370 secs] 7965425K-7790643K(8165824K) icms_dc=3 , 0.6299530 secs]
 [Times: user=0.23 sys=0.03, real=0.63 secs]
 2013-10-07T01:24:43.629+0900: [GC [ParNew: 235800K-26176K(235968K),
 0.3190580 secs] 8000268K-782K(8165824K) icms_dc=3 , 0.3191840 secs]
 [Times: user=0.24 sys=0.02, real=0.32 secs]
 2013-10-07T01:24:56.005+0900: [GC [ParNew: 235848K-26176K(235968K),
 0.4839400 secs] 8035228K-7860300K(8165824K) icms_dc=3 , 0.4840480 secs]
 [Times: user=0.31 sys=0.03, real=0.49 secs]
 2013-10-07T01:25:07.282+0900: [GC [ParNew: 235750K-26176K(235968K),
 0.3423250 secs] 8069875K-7895852K(8165824K) icms_dc=9 , 0.3424380 secs]
 [Times: user=0.21 sys=0.06, real=0.34 secs]
 2013-10-07T01:25:19.853+0900: [GC [ParNew (promotion failed):
 235745K-235745K(235968K), 0.3339710 secs][CMS2013-10-07T01:25:29.750+0900:
 [CMS-concurrent-mark: 12.929/88.767 secs] [Times: user=14.30 sys=3.74,
 real=88.77 secs]
  (concurrent mode failure): 7899125K-2882954K(7929856K), 42.8279810 secs]
 8105422K-2882954K(8165824K), [CMS Perm : 31956K-31861K(53340K)] icms_dc=9
 , 43.1621090 secs] [Times: user=10.40 sys=1.89, real=43.16 secs]
 2013-10-07T01:26:08.288+0900: [GC [1 CMS-initial-mark: 2882954K(7929856K)]
 2978434K(8165824K), 0.0965830 secs] [Times: user=0.04 sys=0.00, real=0.09
 secs]
 Heap
  par new generation   total 235968K, used 197697K [0x000606e0,
 0x000616e0, 0x000616e0)
   eden space 209792K,  94% used [0x000606e0, 0x000612f10718,
 0x000613ae)
   from space 26176K,   0% used [0x00061547, 0x00061547,
 0x000616e0)
   to   space 26176K,   0% used [0x000613ae, 0x000613ae,
 0x00061547)
  concurrent mark-sweep generation total 7929856K, used 2882954K
 [0x000616e0, 0x0007fae0, 0x0007fae0)
  concurrent-mark-sweep perm gen total 53340K, used 31960K
 [0x0007fae0, 0x0007fe217000, 0x0008)

 What is wrong here? please give me some suggestions.


 Sincerely,
 Prakash

You Are Dead Exception due to promotion failure

2013-10-07 Thread prakash kadel 
Hello,
  I am getting this YADE all the time

HBASE_HEAPSIZE=8000

Settings: -ea -XX:+UseConcMarkSweepGC -XX:MaxGCPauseMillis=200
-XX:+HeapDumpOnOutOfMemoryError -XX:+CMSIncrementalMode -XX:+UseParNewGC
-XX:CMSInitiatingOccupancyFraction=50 -XX:+UseCMSInitiatingOccupancyOnly
-XX:NewSize=256m -XX:MaxNewSize=256m

it seems there is promotion failure and the CMS take too long

2013-10-07T01:22:55.784+0900: [GC [ParNew: 235968K-26176K(235968K),
0.3219980 secs] 7709485K-7538063K(8165824K) icms_dc=0 , 0.3221100 secs]
[Times: user=0.27 sys=0.01, real=0.33 secs]
2013-10-07T01:23:07.361+0900: [GC [ParNew: 235842K-26176K(235968K),
0.1899680 secs] 7747729K-7578713K(8165824K) icms_dc=0 , 0.1900700 secs]
[Times: user=0.26 sys=0.02, real=0.19 secs]
2013-10-07T01:23:20.154+0900: [GC [ParNew: 235803K-26176K(235968K),
0.2428200 secs] 7788341K-7615284K(8165824K) icms_dc=0 , 0.2429570 secs]
[Times: user=0.25 sys=0.02, real=0.24 secs]
2013-10-07T01:23:34.594+0900: [GC [ParNew: 235889K-26176K(235968K),
0.2440980 secs] 7824998K-7651179K(8165824K) icms_dc=0 , 0.2442130 secs]
[Times: user=0.20 sys=0.03, real=0.25 secs]
2013-10-07T01:23:47.666+0900: [GC [ParNew: 235906K-26176K(235968K),
0.2998100 secs] 7860909K-7686832K(8165824K) icms_dc=3 , 0.3020280 secs]
[Times: user=0.23 sys=0.04, real=0.30 secs]
2013-10-07T01:23:57.216+0900: [GC [1 CMS-initial-mark: 7660656K(7929856K)]
7788778K(8165824K), 3.7665320 secs] [Times: user=0.07 sys=0.06, real=3.77
secs]
2013-10-07T01:24:05.508+0900: [GC [ParNew: 235811K-26176K(235968K),
0.4632860 secs] 7896468K-7721167K(8165824K) icms_dc=3 , 0.4634100 secs]
[Times: user=0.21 sys=0.03, real=0.46 secs]
2013-10-07T01:24:19.889+0900: [GC [ParNew: 235812K-26176K(235968K),
0.3531980 secs] 7930804K-7755633K(8165824K) icms_dc=3 , 0.3533230 secs]
[Times: user=0.24 sys=0.06, real=0.35 secs]
2013-10-07T01:24:32.832+0900: [GC [ParNew: 235968K-26176K(235968K),
0.6298370 secs] 7965425K-7790643K(8165824K) icms_dc=3 , 0.6299530 secs]
[Times: user=0.23 sys=0.03, real=0.63 secs]
2013-10-07T01:24:43.629+0900: [GC [ParNew: 235800K-26176K(235968K),
0.3190580 secs] 8000268K-782K(8165824K) icms_dc=3 , 0.3191840 secs]
[Times: user=0.24 sys=0.02, real=0.32 secs]
2013-10-07T01:24:56.005+0900: [GC [ParNew: 235848K-26176K(235968K),
0.4839400 secs] 8035228K-7860300K(8165824K) icms_dc=3 , 0.4840480 secs]
[Times: user=0.31 sys=0.03, real=0.49 secs]
2013-10-07T01:25:07.282+0900: [GC [ParNew: 235750K-26176K(235968K),
0.3423250 secs] 8069875K-7895852K(8165824K) icms_dc=9 , 0.3424380 secs]
[Times: user=0.21 sys=0.06, real=0.34 secs]
2013-10-07T01:25:19.853+0900: [GC [ParNew (promotion failed):
235745K-235745K(235968K), 0.3339710 secs][CMS2013-10-07T01:25:29.750+0900:
[CMS-concurrent-mark: 12.929/88.767 secs] [Times: user=14.30 sys=3.74,
real=88.77 secs]
 (concurrent mode failure): 7899125K-2882954K(7929856K), 42.8279810 secs]
8105422K-2882954K(8165824K), [CMS Perm : 31956K-31861K(53340K)] icms_dc=9
, 43.1621090 secs] [Times: user=10.40 sys=1.89, real=43.16 secs]
2013-10-07T01:26:08.288+0900: [GC [1 CMS-initial-mark: 2882954K(7929856K)]
2978434K(8165824K), 0.0965830 secs] [Times: user=0.04 sys=0.00, real=0.09
secs]
Heap
 par new generation   total 235968K, used 197697K [0x000606e0,
0x000616e0, 0x000616e0)
  eden space 209792K,  94% used [0x000606e0, 0x000612f10718,
0x000613ae)
  from space 26176K,   0% used [0x00061547, 0x00061547,
0x000616e0)
  to   space 26176K,   0% used [0x000613ae, 0x000613ae,
0x00061547)
 concurrent mark-sweep generation total 7929856K, used 2882954K
[0x000616e0, 0x0007fae0, 0x0007fae0)
 concurrent-mark-sweep perm gen total 53340K, used 31960K
[0x0007fae0, 0x0007fe217000, 0x0008)

What is wrong here? please give me some suggestions.


Sincerely,
Prakash

Re: You Are Dead Exception due to promotion failure

2013-10-07 Thread Jean-Daniel Cryans 
This line:

[CMS-concurrent-mark: 12.929/88.767 secs] [Times: user=14.30 sys=3.74,
real=88.77
secs]

Is suspicious. Are you swapping?

J-D


On Mon, Oct 7, 2013 at 8:34 AM, prakash kadel prakash.ka...@gmail.comwrote:

 Also,
why is the CMS not kicking in early, i have set XX:+
 UseCMSInitiatingOccupancyOnly???

 Sincerely,
 Prakash


 On Tue, Oct 8, 2013 at 12:32 AM, prakash kadel prakash.ka...@gmail.com
 wrote:

  Hello,
I am getting this YADE all the time
 
  HBASE_HEAPSIZE=8000
 
  Settings: -ea -XX:+UseConcMarkSweepGC -XX:MaxGCPauseMillis=200
  -XX:+HeapDumpOnOutOfMemoryError -XX:+CMSIncrementalMode -XX:+UseParNewGC
  -XX:CMSInitiatingOccupancyFraction=50 -XX:+UseCMSInitiatingOccupancyOnly
  -XX:NewSize=256m -XX:MaxNewSize=256m
 
  it seems there is promotion failure and the CMS take too long
 
  2013-10-07T01:22:55.784+0900: [GC [ParNew: 235968K-26176K(235968K),
  0.3219980 secs] 7709485K-7538063K(8165824K) icms_dc=0 , 0.3221100 secs]
  [Times: user=0.27 sys=0.01, real=0.33 secs]
  2013-10-07T01:23:07.361+0900: [GC [ParNew: 235842K-26176K(235968K),
  0.1899680 secs] 7747729K-7578713K(8165824K) icms_dc=0 , 0.1900700 secs]
  [Times: user=0.26 sys=0.02, real=0.19 secs]
  2013-10-07T01:23:20.154+0900: [GC [ParNew: 235803K-26176K(235968K),
  0.2428200 secs] 7788341K-7615284K(8165824K) icms_dc=0 , 0.2429570 secs]
  [Times: user=0.25 sys=0.02, real=0.24 secs]
  2013-10-07T01:23:34.594+0900: [GC [ParNew: 235889K-26176K(235968K),
  0.2440980 secs] 7824998K-7651179K(8165824K) icms_dc=0 , 0.2442130 secs]
  [Times: user=0.20 sys=0.03, real=0.25 secs]
  2013-10-07T01:23:47.666+0900: [GC [ParNew: 235906K-26176K(235968K),
  0.2998100 secs] 7860909K-7686832K(8165824K) icms_dc=3 , 0.3020280 secs]
  [Times: user=0.23 sys=0.04, real=0.30 secs]
  2013-10-07T01:23:57.216+0900: [GC [1 CMS-initial-mark:
 7660656K(7929856K)]
  7788778K(8165824K), 3.7665320 secs] [Times: user=0.07 sys=0.06, real=3.77
  secs]
  2013-10-07T01:24:05.508+0900: [GC [ParNew: 235811K-26176K(235968K),
  0.4632860 secs] 7896468K-7721167K(8165824K) icms_dc=3 , 0.4634100 secs]
  [Times: user=0.21 sys=0.03, real=0.46 secs]
  2013-10-07T01:24:19.889+0900: [GC [ParNew: 235812K-26176K(235968K),
  0.3531980 secs] 7930804K-7755633K(8165824K) icms_dc=3 , 0.3533230 secs]
  [Times: user=0.24 sys=0.06, real=0.35 secs]
  2013-10-07T01:24:32.832+0900: [GC [ParNew: 235968K-26176K(235968K),
  0.6298370 secs] 7965425K-7790643K(8165824K) icms_dc=3 , 0.6299530 secs]
  [Times: user=0.23 sys=0.03, real=0.63 secs]
  2013-10-07T01:24:43.629+0900: [GC [ParNew: 235800K-26176K(235968K),
  0.3190580 secs] 8000268K-782K(8165824K) icms_dc=3 , 0.3191840 secs]
  [Times: user=0.24 sys=0.02, real=0.32 secs]
  2013-10-07T01:24:56.005+0900: [GC [ParNew: 235848K-26176K(235968K),
  0.4839400 secs] 8035228K-7860300K(8165824K) icms_dc=3 , 0.4840480 secs]
  [Times: user=0.31 sys=0.03, real=0.49 secs]
  2013-10-07T01:25:07.282+0900: [GC [ParNew: 235750K-26176K(235968K),
  0.3423250 secs] 8069875K-7895852K(8165824K) icms_dc=9 , 0.3424380 secs]
  [Times: user=0.21 sys=0.06, real=0.34 secs]
  2013-10-07T01:25:19.853+0900: [GC [ParNew (promotion failed):
  235745K-235745K(235968K), 0.3339710
 secs][CMS2013-10-07T01:25:29.750+0900:
  [CMS-concurrent-mark: 12.929/88.767 secs] [Times: user=14.30 sys=3.74,
  real=88.77 secs]
   (concurrent mode failure): 7899125K-2882954K(7929856K), 42.8279810
 secs]
  8105422K-2882954K(8165824K), [CMS Perm : 31956K-31861K(53340K)]
 icms_dc=9
  , 43.1621090 secs] [Times: user=10.40 sys=1.89, real=43.16 secs]
  2013-10-07T01:26:08.288+0900: [GC [1 CMS-initial-mark:
 2882954K(7929856K)]
  2978434K(8165824K), 0.0965830 secs] [Times: user=0.04 sys=0.00, real=0.09
  secs]
  Heap
   par new generation   total 235968K, used 197697K [0x000606e0,
  0x000616e0, 0x000616e0)
eden space 209792K,  94% used [0x000606e0, 0x000612f10718,
  0x000613ae)
from space 26176K,   0% used [0x00061547, 0x00061547,
  0x000616e0)
to   space 26176K,   0% used [0x000613ae, 0x000613ae,
  0x00061547)
   concurrent mark-sweep generation total 7929856K, used 2882954K
  [0x000616e0, 0x0007fae0, 0x0007fae0)
   concurrent-mark-sweep perm gen total 53340K, used 31960K
  [0x0007fae0, 0x0007fe217000, 0x0008)
 
  What is wrong here? please give me some suggestions.
 
 
  Sincerely,
  Prakash

Re: You Are Dead Exception due to promotion failure

2013-10-07 Thread prakash kadel 
thanks,

yup, it seems so. I have 48 gb memory. i see it swaps at that point.

btw, why is the CMS not kicking in early? do you have any idea?

sincerely



On Tue, Oct 8, 2013 at 3:00 AM, Jean-Daniel Cryans jdcry...@apache.orgwrote:

 This line:

 [CMS-concurrent-mark: 12.929/88.767 secs] [Times: user=14.30 sys=3.74,
 real=88.77
 secs]

 Is suspicious. Are you swapping?

 J-D


 On Mon, Oct 7, 2013 at 8:34 AM, prakash kadel prakash.ka...@gmail.com
 wrote:

  Also,
 why is the CMS not kicking in early, i have set XX:+
  UseCMSInitiatingOccupancyOnly???
 
  Sincerely,
  Prakash
 
 
  On Tue, Oct 8, 2013 at 12:32 AM, prakash kadel prakash.ka...@gmail.com
  wrote:
 
   Hello,
 I am getting this YADE all the time
  
   HBASE_HEAPSIZE=8000
  
   Settings: -ea -XX:+UseConcMarkSweepGC -XX:MaxGCPauseMillis=200
   -XX:+HeapDumpOnOutOfMemoryError -XX:+CMSIncrementalMode
 -XX:+UseParNewGC
   -XX:CMSInitiatingOccupancyFraction=50
 -XX:+UseCMSInitiatingOccupancyOnly
   -XX:NewSize=256m -XX:MaxNewSize=256m
  
   it seems there is promotion failure and the CMS take too long
  
   2013-10-07T01:22:55.784+0900: [GC [ParNew: 235968K-26176K(235968K),
   0.3219980 secs] 7709485K-7538063K(8165824K) icms_dc=0 , 0.3221100
 secs]
   [Times: user=0.27 sys=0.01, real=0.33 secs]
   2013-10-07T01:23:07.361+0900: [GC [ParNew: 235842K-26176K(235968K),
   0.1899680 secs] 7747729K-7578713K(8165824K) icms_dc=0 , 0.1900700
 secs]
   [Times: user=0.26 sys=0.02, real=0.19 secs]
   2013-10-07T01:23:20.154+0900: [GC [ParNew: 235803K-26176K(235968K),
   0.2428200 secs] 7788341K-7615284K(8165824K) icms_dc=0 , 0.2429570
 secs]
   [Times: user=0.25 sys=0.02, real=0.24 secs]
   2013-10-07T01:23:34.594+0900: [GC [ParNew: 235889K-26176K(235968K),
   0.2440980 secs] 7824998K-7651179K(8165824K) icms_dc=0 , 0.2442130
 secs]
   [Times: user=0.20 sys=0.03, real=0.25 secs]
   2013-10-07T01:23:47.666+0900: [GC [ParNew: 235906K-26176K(235968K),
   0.2998100 secs] 7860909K-7686832K(8165824K) icms_dc=3 , 0.3020280
 secs]
   [Times: user=0.23 sys=0.04, real=0.30 secs]
   2013-10-07T01:23:57.216+0900: [GC [1 CMS-initial-mark:
  7660656K(7929856K)]
   7788778K(8165824K), 3.7665320 secs] [Times: user=0.07 sys=0.06,
 real=3.77
   secs]
   2013-10-07T01:24:05.508+0900: [GC [ParNew: 235811K-26176K(235968K),
   0.4632860 secs] 7896468K-7721167K(8165824K) icms_dc=3 , 0.4634100
 secs]
   [Times: user=0.21 sys=0.03, real=0.46 secs]
   2013-10-07T01:24:19.889+0900: [GC [ParNew: 235812K-26176K(235968K),
   0.3531980 secs] 7930804K-7755633K(8165824K) icms_dc=3 , 0.3533230
 secs]
   [Times: user=0.24 sys=0.06, real=0.35 secs]
   2013-10-07T01:24:32.832+0900: [GC [ParNew: 235968K-26176K(235968K),
   0.6298370 secs] 7965425K-7790643K(8165824K) icms_dc=3 , 0.6299530
 secs]
   [Times: user=0.23 sys=0.03, real=0.63 secs]
   2013-10-07T01:24:43.629+0900: [GC [ParNew: 235800K-26176K(235968K),
   0.3190580 secs] 8000268K-782K(8165824K) icms_dc=3 , 0.3191840
 secs]
   [Times: user=0.24 sys=0.02, real=0.32 secs]
   2013-10-07T01:24:56.005+0900: [GC [ParNew: 235848K-26176K(235968K),
   0.4839400 secs] 8035228K-7860300K(8165824K) icms_dc=3 , 0.4840480
 secs]
   [Times: user=0.31 sys=0.03, real=0.49 secs]
   2013-10-07T01:25:07.282+0900: [GC [ParNew: 235750K-26176K(235968K),
   0.3423250 secs] 8069875K-7895852K(8165824K) icms_dc=9 , 0.3424380
 secs]
   [Times: user=0.21 sys=0.06, real=0.34 secs]
   2013-10-07T01:25:19.853+0900: [GC [ParNew (promotion failed):
   235745K-235745K(235968K), 0.3339710
  secs][CMS2013-10-07T01:25:29.750+0900:
   [CMS-concurrent-mark: 12.929/88.767 secs] [Times: user=14.30 sys=3.74,
   real=88.77 secs]
(concurrent mode failure): 7899125K-2882954K(7929856K), 42.8279810
  secs]
   8105422K-2882954K(8165824K), [CMS Perm : 31956K-31861K(53340K)]
  icms_dc=9
   , 43.1621090 secs] [Times: user=10.40 sys=1.89, real=43.16 secs]
   2013-10-07T01:26:08.288+0900: [GC [1 CMS-initial-mark:
  2882954K(7929856K)]
   2978434K(8165824K), 0.0965830 secs] [Times: user=0.04 sys=0.00,
 real=0.09
   secs]
   Heap
par new generation   total 235968K, used 197697K [0x000606e0,
   0x000616e0, 0x000616e0)
 eden space 209792K,  94% used [0x000606e0,
 0x000612f10718,
   0x000613ae)
 from space 26176K,   0% used [0x00061547, 0x00061547,
   0x000616e0)
 to   space 26176K,   0% used [0x000613ae, 0x000613ae,
   0x00061547)
concurrent mark-sweep generation total 7929856K, used 2882954K
   [0x000616e0, 0x0007fae0, 0x0007fae0)
concurrent-mark-sweep perm gen total 53340K, used 31960K
   [0x0007fae0, 0x0007fe217000, 0x0008)
  
   What is wrong here? please give me some suggestions.
  
  
   Sincerely,
   Prakash

Re: You Are Dead Exception due to promotion failure

2013-10-07 Thread prakash kadel 
BTW,
  if i disable the swap at all. What will happen in the above situation?
currently it starts swapping at 90%

Sincerely


On Tue, Oct 8, 2013 at 7:09 AM, prakash kadel prakash.ka...@gmail.comwrote:

 thanks,

 yup, it seems so. I have 48 gb memory. i see it swaps at that point.

 btw, why is the CMS not kicking in early? do you have any idea?

 sincerely



 On Tue, Oct 8, 2013 at 3:00 AM, Jean-Daniel Cryans jdcry...@apache.orgwrote:

 This line:

 [CMS-concurrent-mark: 12.929/88.767 secs] [Times: user=14.30 sys=3.74,
 real=88.77
 secs]

 Is suspicious. Are you swapping?

 J-D


 On Mon, Oct 7, 2013 at 8:34 AM, prakash kadel prakash.ka...@gmail.com
 wrote:

  Also,
 why is the CMS not kicking in early, i have set XX:+
  UseCMSInitiatingOccupancyOnly???
 
  Sincerely,
  Prakash
 
 
  On Tue, Oct 8, 2013 at 12:32 AM, prakash kadel prakash.ka...@gmail.com
  wrote:
 
   Hello,
 I am getting this YADE all the time
  
   HBASE_HEAPSIZE=8000
  
   Settings: -ea -XX:+UseConcMarkSweepGC -XX:MaxGCPauseMillis=200
   -XX:+HeapDumpOnOutOfMemoryError -XX:+CMSIncrementalMode
 -XX:+UseParNewGC
   -XX:CMSInitiatingOccupancyFraction=50
 -XX:+UseCMSInitiatingOccupancyOnly
   -XX:NewSize=256m -XX:MaxNewSize=256m
  
   it seems there is promotion failure and the CMS take too long
  
   2013-10-07T01:22:55.784+0900: [GC [ParNew: 235968K-26176K(235968K),
   0.3219980 secs] 7709485K-7538063K(8165824K) icms_dc=0 , 0.3221100
 secs]
   [Times: user=0.27 sys=0.01, real=0.33 secs]
   2013-10-07T01:23:07.361+0900: [GC [ParNew: 235842K-26176K(235968K),
   0.1899680 secs] 7747729K-7578713K(8165824K) icms_dc=0 , 0.1900700
 secs]
   [Times: user=0.26 sys=0.02, real=0.19 secs]
   2013-10-07T01:23:20.154+0900: [GC [ParNew: 235803K-26176K(235968K),
   0.2428200 secs] 7788341K-7615284K(8165824K) icms_dc=0 , 0.2429570
 secs]
   [Times: user=0.25 sys=0.02, real=0.24 secs]
   2013-10-07T01:23:34.594+0900: [GC [ParNew: 235889K-26176K(235968K),
   0.2440980 secs] 7824998K-7651179K(8165824K) icms_dc=0 , 0.2442130
 secs]
   [Times: user=0.20 sys=0.03, real=0.25 secs]
   2013-10-07T01:23:47.666+0900: [GC [ParNew: 235906K-26176K(235968K),
   0.2998100 secs] 7860909K-7686832K(8165824K) icms_dc=3 , 0.3020280
 secs]
   [Times: user=0.23 sys=0.04, real=0.30 secs]
   2013-10-07T01:23:57.216+0900: [GC [1 CMS-initial-mark:
  7660656K(7929856K)]
   7788778K(8165824K), 3.7665320 secs] [Times: user=0.07 sys=0.06,
 real=3.77
   secs]
   2013-10-07T01:24:05.508+0900: [GC [ParNew: 235811K-26176K(235968K),
   0.4632860 secs] 7896468K-7721167K(8165824K) icms_dc=3 , 0.4634100
 secs]
   [Times: user=0.21 sys=0.03, real=0.46 secs]
   2013-10-07T01:24:19.889+0900: [GC [ParNew: 235812K-26176K(235968K),
   0.3531980 secs] 7930804K-7755633K(8165824K) icms_dc=3 , 0.3533230
 secs]
   [Times: user=0.24 sys=0.06, real=0.35 secs]
   2013-10-07T01:24:32.832+0900: [GC [ParNew: 235968K-26176K(235968K),
   0.6298370 secs] 7965425K-7790643K(8165824K) icms_dc=3 , 0.6299530
 secs]
   [Times: user=0.23 sys=0.03, real=0.63 secs]
   2013-10-07T01:24:43.629+0900: [GC [ParNew: 235800K-26176K(235968K),
   0.3190580 secs] 8000268K-782K(8165824K) icms_dc=3 , 0.3191840
 secs]
   [Times: user=0.24 sys=0.02, real=0.32 secs]
   2013-10-07T01:24:56.005+0900: [GC [ParNew: 235848K-26176K(235968K),
   0.4839400 secs] 8035228K-7860300K(8165824K) icms_dc=3 , 0.4840480
 secs]
   [Times: user=0.31 sys=0.03, real=0.49 secs]
   2013-10-07T01:25:07.282+0900: [GC [ParNew: 235750K-26176K(235968K),
   0.3423250 secs] 8069875K-7895852K(8165824K) icms_dc=9 , 0.3424380
 secs]
   [Times: user=0.21 sys=0.06, real=0.34 secs]
   2013-10-07T01:25:19.853+0900: [GC [ParNew (promotion failed):
   235745K-235745K(235968K), 0.3339710
  secs][CMS2013-10-07T01:25:29.750+0900:
   [CMS-concurrent-mark: 12.929/88.767 secs] [Times: user=14.30 sys=3.74,
   real=88.77 secs]
(concurrent mode failure): 7899125K-2882954K(7929856K), 42.8279810
  secs]
   8105422K-2882954K(8165824K), [CMS Perm : 31956K-31861K(53340K)]
  icms_dc=9
   , 43.1621090 secs] [Times: user=10.40 sys=1.89, real=43.16 secs]
   2013-10-07T01:26:08.288+0900: [GC [1 CMS-initial-mark:
  2882954K(7929856K)]
   2978434K(8165824K), 0.0965830 secs] [Times: user=0.04 sys=0.00,
 real=0.09
   secs]
   Heap
par new generation   total 235968K, used 197697K [0x000606e0,
   0x000616e0, 0x000616e0)
 eden space 209792K,  94% used [0x000606e0,
 0x000612f10718,
   0x000613ae)
 from space 26176K,   0% used [0x00061547,
 0x00061547,
   0x000616e0)
 to   space 26176K,   0% used [0x000613ae,
 0x000613ae,
   0x00061547)
concurrent mark-sweep generation total 7929856K, used 2882954K
   [0x000616e0, 0x0007fae0, 0x0007fae0)
concurrent-mark-sweep perm gen total 53340K, used 31960K
   [0x0007fae0, 0x0007fe217000, 0x0008)
  
   What is wrong here? please give me some suggestions.
  
  
   Sincerely,
   Prakash

Re: You Are Dead Exception due to promotion failure

2013-10-07 Thread Jean-Daniel Cryans 
Swapping and Java simply don't go well together. You need to ensure that
the committed memory is smaller than the available memory. Also see
http://hbase.apache.org/book.html#perf.os.swap

I haven't looked closely at your GC output but even if CMS was kicking as
early as it's supposed to, the fact that you are swapping might just screw
up everything.

J-D


On Mon, Oct 7, 2013 at 3:13 PM, prakash kadel prakash.ka...@gmail.comwrote:

 BTW,
   if i disable the swap at all. What will happen in the above situation?
 currently it starts swapping at 90%

 Sincerely


 On Tue, Oct 8, 2013 at 7:09 AM, prakash kadel prakash.ka...@gmail.com
 wrote:

  thanks,
 
  yup, it seems so. I have 48 gb memory. i see it swaps at that point.
 
  btw, why is the CMS not kicking in early? do you have any idea?
 
  sincerely
 
 
 
  On Tue, Oct 8, 2013 at 3:00 AM, Jean-Daniel Cryans jdcry...@apache.org
 wrote:
 
  This line:
 
  [CMS-concurrent-mark: 12.929/88.767 secs] [Times: user=14.30 sys=3.74,
  real=88.77
  secs]
 
  Is suspicious. Are you swapping?
 
  J-D
 
 
  On Mon, Oct 7, 2013 at 8:34 AM, prakash kadel prakash.ka...@gmail.com
  wrote:
 
   Also,
  why is the CMS not kicking in early, i have set XX:+
   UseCMSInitiatingOccupancyOnly???
  
   Sincerely,
   Prakash
  
  
   On Tue, Oct 8, 2013 at 12:32 AM, prakash kadel 
 prakash.ka...@gmail.com
   wrote:
  
Hello,
  I am getting this YADE all the time
   
HBASE_HEAPSIZE=8000
   
Settings: -ea -XX:+UseConcMarkSweepGC -XX:MaxGCPauseMillis=200
-XX:+HeapDumpOnOutOfMemoryError -XX:+CMSIncrementalMode
  -XX:+UseParNewGC
-XX:CMSInitiatingOccupancyFraction=50
  -XX:+UseCMSInitiatingOccupancyOnly
-XX:NewSize=256m -XX:MaxNewSize=256m
   
it seems there is promotion failure and the CMS take too long
   
2013-10-07T01:22:55.784+0900: [GC [ParNew: 235968K-26176K(235968K),
0.3219980 secs] 7709485K-7538063K(8165824K) icms_dc=0 , 0.3221100
  secs]
[Times: user=0.27 sys=0.01, real=0.33 secs]
2013-10-07T01:23:07.361+0900: [GC [ParNew: 235842K-26176K(235968K),
0.1899680 secs] 7747729K-7578713K(8165824K) icms_dc=0 , 0.1900700
  secs]
[Times: user=0.26 sys=0.02, real=0.19 secs]
2013-10-07T01:23:20.154+0900: [GC [ParNew: 235803K-26176K(235968K),
0.2428200 secs] 7788341K-7615284K(8165824K) icms_dc=0 , 0.2429570
  secs]
[Times: user=0.25 sys=0.02, real=0.24 secs]
2013-10-07T01:23:34.594+0900: [GC [ParNew: 235889K-26176K(235968K),
0.2440980 secs] 7824998K-7651179K(8165824K) icms_dc=0 , 0.2442130
  secs]
[Times: user=0.20 sys=0.03, real=0.25 secs]
2013-10-07T01:23:47.666+0900: [GC [ParNew: 235906K-26176K(235968K),
0.2998100 secs] 7860909K-7686832K(8165824K) icms_dc=3 , 0.3020280
  secs]
[Times: user=0.23 sys=0.04, real=0.30 secs]
2013-10-07T01:23:57.216+0900: [GC [1 CMS-initial-mark:
   7660656K(7929856K)]
7788778K(8165824K), 3.7665320 secs] [Times: user=0.07 sys=0.06,
  real=3.77
secs]
2013-10-07T01:24:05.508+0900: [GC [ParNew: 235811K-26176K(235968K),
0.4632860 secs] 7896468K-7721167K(8165824K) icms_dc=3 , 0.4634100
  secs]
[Times: user=0.21 sys=0.03, real=0.46 secs]
2013-10-07T01:24:19.889+0900: [GC [ParNew: 235812K-26176K(235968K),
0.3531980 secs] 7930804K-7755633K(8165824K) icms_dc=3 , 0.3533230
  secs]
[Times: user=0.24 sys=0.06, real=0.35 secs]
2013-10-07T01:24:32.832+0900: [GC [ParNew: 235968K-26176K(235968K),
0.6298370 secs] 7965425K-7790643K(8165824K) icms_dc=3 , 0.6299530
  secs]
[Times: user=0.23 sys=0.03, real=0.63 secs]
2013-10-07T01:24:43.629+0900: [GC [ParNew: 235800K-26176K(235968K),
0.3190580 secs] 8000268K-782K(8165824K) icms_dc=3 , 0.3191840
  secs]
[Times: user=0.24 sys=0.02, real=0.32 secs]
2013-10-07T01:24:56.005+0900: [GC [ParNew: 235848K-26176K(235968K),
0.4839400 secs] 8035228K-7860300K(8165824K) icms_dc=3 , 0.4840480
  secs]
[Times: user=0.31 sys=0.03, real=0.49 secs]
2013-10-07T01:25:07.282+0900: [GC [ParNew: 235750K-26176K(235968K),
0.3423250 secs] 8069875K-7895852K(8165824K) icms_dc=9 , 0.3424380
  secs]
[Times: user=0.21 sys=0.06, real=0.34 secs]
2013-10-07T01:25:19.853+0900: [GC [ParNew (promotion failed):
235745K-235745K(235968K), 0.3339710
   secs][CMS2013-10-07T01:25:29.750+0900:
[CMS-concurrent-mark: 12.929/88.767 secs] [Times: user=14.30
 sys=3.74,
real=88.77 secs]
 (concurrent mode failure): 7899125K-2882954K(7929856K), 42.8279810
   secs]
8105422K-2882954K(8165824K), [CMS Perm : 31956K-31861K(53340K)]
   icms_dc=9
, 43.1621090 secs] [Times: user=10.40 sys=1.89, real=43.16 secs]
2013-10-07T01:26:08.288+0900: [GC [1 CMS-initial-mark:
   2882954K(7929856K)]
2978434K(8165824K), 0.0965830 secs] [Times: user=0.04 sys=0.00,
  real=0.09
secs]
Heap
 par new generation   total 235968K, used 197697K
 [0x000606e0,
0x000616e0, 0x000616e0)
  eden space 209792K,  94% used [0x000606e0,

答复: You Are Dead Exception due to promotion failure

2013-10-07 Thread 谢良 
The OS will pick up a process to kill according to an internal oom 
score,weight...(very likely is HBase if it ran long time and consumed lots of 
memory)

发件人: prakash kadel [prakash.ka...@gmail.com]
发送时间: 2013年10月8日 6:13
收件人: user@hbase.apache.org
主题: Re: You Are Dead Exception due to promotion failure

BTW,
  if i disable the swap at all. What will happen in the above situation?
currently it starts swapping at 90%

Sincerely


On Tue, Oct 8, 2013 at 7:09 AM, prakash kadel prakash.ka...@gmail.comwrote:

 thanks,

 yup, it seems so. I have 48 gb memory. i see it swaps at that point.

 btw, why is the CMS not kicking in early? do you have any idea?

 sincerely



 On Tue, Oct 8, 2013 at 3:00 AM, Jean-Daniel Cryans jdcry...@apache.orgwrote:

 This line:

 [CMS-concurrent-mark: 12.929/88.767 secs] [Times: user=14.30 sys=3.74,
 real=88.77
 secs]

 Is suspicious. Are you swapping?

 J-D


 On Mon, Oct 7, 2013 at 8:34 AM, prakash kadel prakash.ka...@gmail.com
 wrote:

  Also,
 why is the CMS not kicking in early, i have set XX:+
  UseCMSInitiatingOccupancyOnly???
 
  Sincerely,
  Prakash
 
 
  On Tue, Oct 8, 2013 at 12:32 AM, prakash kadel prakash.ka...@gmail.com
  wrote:
 
   Hello,
 I am getting this YADE all the time
  
   HBASE_HEAPSIZE=8000
  
   Settings: -ea -XX:+UseConcMarkSweepGC -XX:MaxGCPauseMillis=200
   -XX:+HeapDumpOnOutOfMemoryError -XX:+CMSIncrementalMode
 -XX:+UseParNewGC
   -XX:CMSInitiatingOccupancyFraction=50
 -XX:+UseCMSInitiatingOccupancyOnly
   -XX:NewSize=256m -XX:MaxNewSize=256m
  
   it seems there is promotion failure and the CMS take too long
  
   2013-10-07T01:22:55.784+0900: [GC [ParNew: 235968K-26176K(235968K),
   0.3219980 secs] 7709485K-7538063K(8165824K) icms_dc=0 , 0.3221100
 secs]
   [Times: user=0.27 sys=0.01, real=0.33 secs]
   2013-10-07T01:23:07.361+0900: [GC [ParNew: 235842K-26176K(235968K),
   0.1899680 secs] 7747729K-7578713K(8165824K) icms_dc=0 , 0.1900700
 secs]
   [Times: user=0.26 sys=0.02, real=0.19 secs]
   2013-10-07T01:23:20.154+0900: [GC [ParNew: 235803K-26176K(235968K),
   0.2428200 secs] 7788341K-7615284K(8165824K) icms_dc=0 , 0.2429570
 secs]
   [Times: user=0.25 sys=0.02, real=0.24 secs]
   2013-10-07T01:23:34.594+0900: [GC [ParNew: 235889K-26176K(235968K),
   0.2440980 secs] 7824998K-7651179K(8165824K) icms_dc=0 , 0.2442130
 secs]
   [Times: user=0.20 sys=0.03, real=0.25 secs]
   2013-10-07T01:23:47.666+0900: [GC [ParNew: 235906K-26176K(235968K),
   0.2998100 secs] 7860909K-7686832K(8165824K) icms_dc=3 , 0.3020280
 secs]
   [Times: user=0.23 sys=0.04, real=0.30 secs]
   2013-10-07T01:23:57.216+0900: [GC [1 CMS-initial-mark:
  7660656K(7929856K)]
   7788778K(8165824K), 3.7665320 secs] [Times: user=0.07 sys=0.06,
 real=3.77
   secs]
   2013-10-07T01:24:05.508+0900: [GC [ParNew: 235811K-26176K(235968K),
   0.4632860 secs] 7896468K-7721167K(8165824K) icms_dc=3 , 0.4634100
 secs]
   [Times: user=0.21 sys=0.03, real=0.46 secs]
   2013-10-07T01:24:19.889+0900: [GC [ParNew: 235812K-26176K(235968K),
   0.3531980 secs] 7930804K-7755633K(8165824K) icms_dc=3 , 0.3533230
 secs]
   [Times: user=0.24 sys=0.06, real=0.35 secs]
   2013-10-07T01:24:32.832+0900: [GC [ParNew: 235968K-26176K(235968K),
   0.6298370 secs] 7965425K-7790643K(8165824K) icms_dc=3 , 0.6299530
 secs]
   [Times: user=0.23 sys=0.03, real=0.63 secs]
   2013-10-07T01:24:43.629+0900: [GC [ParNew: 235800K-26176K(235968K),
   0.3190580 secs] 8000268K-782K(8165824K) icms_dc=3 , 0.3191840
 secs]
   [Times: user=0.24 sys=0.02, real=0.32 secs]
   2013-10-07T01:24:56.005+0900: [GC [ParNew: 235848K-26176K(235968K),
   0.4839400 secs] 8035228K-7860300K(8165824K) icms_dc=3 , 0.4840480
 secs]
   [Times: user=0.31 sys=0.03, real=0.49 secs]
   2013-10-07T01:25:07.282+0900: [GC [ParNew: 235750K-26176K(235968K),
   0.3423250 secs] 8069875K-7895852K(8165824K) icms_dc=9 , 0.3424380
 secs]
   [Times: user=0.21 sys=0.06, real=0.34 secs]
   2013-10-07T01:25:19.853+0900: [GC [ParNew (promotion failed):
   235745K-235745K(235968K), 0.3339710
  secs][CMS2013-10-07T01:25:29.750+0900:
   [CMS-concurrent-mark: 12.929/88.767 secs] [Times: user=14.30 sys=3.74,
   real=88.77 secs]
(concurrent mode failure): 7899125K-2882954K(7929856K), 42.8279810
  secs]
   8105422K-2882954K(8165824K), [CMS Perm : 31956K-31861K(53340K)]
  icms_dc=9
   , 43.1621090 secs] [Times: user=10.40 sys=1.89, real=43.16 secs]
   2013-10-07T01:26:08.288+0900: [GC [1 CMS-initial-mark:
  2882954K(7929856K)]
   2978434K(8165824K), 0.0965830 secs] [Times: user=0.04 sys=0.00,
 real=0.09
   secs]
   Heap
par new generation   total 235968K, used 197697K [0x000606e0,
   0x000616e0, 0x000616e0)
 eden space 209792K,  94% used [0x000606e0,
 0x000612f10718,
   0x000613ae)
 from space 26176K,   0% used [0x00061547,
 0x00061547,
   0x000616e0)
 to   space 26176K,   0% used [0x000613ae,
 0x000613ae,
   0x00061547