GC storming, possible memory leak
---------------------------------
Key: CASSANDRA-1014
URL: https://issues.apache.org/jira/browse/CASSANDRA-1014
Project: Cassandra
Issue Type: Bug
Affects Versions: 0.6.1
Environment: debian lenny amd64 OpenJDK 64-Bit Server VM (build
1.6.0_0-b11, mixed mode)
Reporter: Brandon Williams
Attachments: 724-0001.png
There appears to be a GC issue due to memory pressure in the 0.6 branch. You
can see this by starting the server and performing many inserts. Quickly the
jvm will consume most of its heap, and pauses for stop-the-world GC will begin.
With verbose GC turned on, this can be observed as follows:
[GC [ParNew (promotion failed): 79703K->79703K(84544K), 0.0622980
secs][CMS[CMS-concurrent-mark: 3.678/5.031 secs] [Times: user=10.35 sys=4.22,
real=5.03 secs]
(concurrent mode failure): 944529K->492222K(963392K), 2.8264480 secs]
990745K->492222K(1047936K), 2.8890500 secs] [Times: user=2.90 sys=0.04,
real=2.90 secs]
After enough inserts (around 75-100 million) the server will GC storm and then
OOM.
jbellis and I narrowed this down to patch 0001 in CASSANDRA-724. Switching LBQ
with ABQ made no difference, however using batch mode instead of periodic for
the commitlog does prevent the issue from occurring. The attached screenshot
show the heap usage in jconsole first when the issue is exhibiting, a restart,
and then the same amount of inserts when it does not.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
