[
https://issues.apache.org/jira/browse/CASSANDRA-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002783#comment-13002783
]
Sébastien Giroux commented on CASSANDRA-2273:
---------------------------------------------
And here is one more from a different node:
ERROR 18:51:36,970 Fatal exception in thread Thread[HintedHandoff:1,1,main]
java.lang.RuntimeException: java.lang.RuntimeException:
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap
space
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: java.lang.RuntimeException: java.util.concurrent.ExecutionException:
java.lang.OutOfMemoryError: Java heap space
at
org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:356)
at
org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
at
org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:409)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
... 3 more
Caused by: java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError:
Java heap space
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
at java.util.concurrent.FutureTask.get(FutureTask.java:111)
at
org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:352)
... 6 more
Caused by: java.lang.OutOfMemoryError: Java heap space
at
org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:121)
at
org.apache.cassandra.io.sstable.SSTableWriter.<init>(SSTableWriter.java:78)
at
org.apache.cassandra.db.ColumnFamilyStore.createCompactionWriter(ColumnFamilyStore.java:2145)
at
org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:448)
at
org.apache.cassandra.db.CompactionManager$4.call(CompactionManager.java:240)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
... 3 more
ERROR 18:51:36,971 Fatal exception in thread Thread[FlushWriter:1,5,main]
java.lang.OutOfMemoryError: Java heap space
at
org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:121)
at
org.apache.cassandra.io.sstable.SSTableWriter.<init>(SSTableWriter.java:78)
at
org.apache.cassandra.db.ColumnFamilyStore.createFlushWriter(ColumnFamilyStore.java:2140)
at
org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:158)
at org.apache.cassandra.db.Memtable.access$000(Memtable.java:51)
at org.apache.cassandra.db.Memtable$1.runMayThrow(Memtable.java:176)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
> Possible Memory leak
> --------------------
>
> Key: CASSANDRA-2273
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2273
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Sébastien Giroux
> Priority: Critical
> Fix For: 0.7.4
>
> Attachments: heap_peak_OOM.PNG, jconsole_OOM.PNG
>
>
> I have a few problematic nodes in my cluster that will crash OutOfMemory very
> often. This is Cassandra 0.7.3 downloaded from Hudson.
> Heap size is 6GB, server memory is 8GB.
> Memtable are flushed at 64MB, I have 5 CFs.
> FlushLargestMemtablesAt is set at 0.8 but doesn't help with this issue.
> I will attach a screenshot showing my issue. There is no compaction going on
> when the heap usage start increasing like crazy.
> It could be a configuration issue but it kinda looks like a bug to me.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
