cassandra can not start，OOM
---------------------------
Key: CASSANDRA-3535
URL: https://issues.apache.org/jira/browse/CASSANDRA-3535
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.0.3
Environment: centos6.0
java version "1.6.0_17"
OpenJDK Runtime Environment (IcedTea6 1.7.4) (rhel-1.21.b17.el6-x86_64)
OpenJDK 64-Bit Server VM (build 14.0-b16, mixed mode)
filesystem：ext4
memory 4G
xmx=2G xmn=200M
flush_largest_memtables_at: 0.75
memtable_total_space_in_mb: 128
commitlog_total_space_in_mb: 2048
Reporter: zhenyu
Fix For: 1.0.4
INFO [main] 2011-11-27 16:36:31,337 CommitLogSegment.java (line 60) Creating
new commitlog segment /data/dtest/commitlog/CommitLog-1322382991337.log
INFO [main] 2011-11-27 16:36:31,345 CommitLog.java (line 171) Replaying
/data/dtest/commitlog/CommitLog-1322380100367.log,
/data/dtest/commitlog/CommitLog-1322381434531.log,
/data/dtest/commitlog/CommitLog-1322382917872.log
INFO [main] 2011-11-27 16:36:32,842 CommitLog.java (line 354) Finished reading
/data/dtest/commitlog/CommitLog-1322380100367.log
INFO [main] 2011-11-27 16:36:32,843 CommitLog.java (line 354) Finished reading
/data/dtest/commitlog/CommitLog-1322381434531.log
INFO [main] 2011-11-27 16:36:32,843 CommitLog.java (line 354) Finished reading
/data/dtest/commitlog/CommitLog-1322382917872.log
WARN [ScheduledTasks:1] 2011-11-27 16:36:52,874 GCInspector.java (line 140)
Heap is 0.9563799224191398 full. You may need to reduce memtable and/or cache
sizes. Cassandra is now reducing cache sizes to free up memory. Adjust
reduce_cache_sizes_at threshold in cassandra.yaml if you don't want Cassandra
to do this automatically
WARN [ScheduledTasks:1] 2011-11-27 16:36:52,876 AutoSavingCache.java (line
187) Reducing 0126 KeyCache capacity from 200000 to 0 to reduce memory pressure
WARN [ScheduledTasks:1] 2011-11-27 16:36:52,877 AutoSavingCache.java (line
187) Reducing 0118 KeyCache capacity from 200000 to 4691 to reduce memory
pressure
ERROR [MutationStage:7] 2011-11-27 16:36:52,894 AbstractCassandraDaemon.java
(line 133) Fatal exception in thread Thread[MutationStage:7,5,main]
java.lang.OutOfMemoryError: Java heap space
at
org.apache.cassandra.io.util.RandomAccessReader.<init>(RandomAccessReader.java:75)
at
org.apache.cassandra.io.compress.CompressedRandomAccessReader.<init>(CompressedRandomAccessReader.java:62)
at
org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:44)
at
org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:39)
at
org.apache.cassandra.io.util.CompressedSegmentedFile.getSegment(CompressedSegmentedFile.java:63)
at
org.apache.cassandra.io.sstable.SSTableReader.getFileDataInput(SSTableReader.java:790)
at
org.apache.cassandra.db.columniterator.SSTableNamesIterator.<init>(SSTableNamesIterator.java:61)
at
org.apache.cassandra.db.filter.NamesQueryFilter.getSSTableColumnIterator(NamesQueryFilter.java:60)
at
org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:78)
at
org.apache.cassandra.db.CollationController.collectTimeOrderedData(CollationController.java:114)
at
org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:62)
at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1274)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1160)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1122)
at
org.apache.cassandra.db.Table.readCurrentIndexedColumns(Table.java:504)
at org.apache.cassandra.db.Table.apply(Table.java:441)
at
org.apache.cassandra.db.commitlog.CommitLog$2.runMayThrow(CommitLog.java:338)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
ERROR [MutationStage:8] 2011-11-27 16:36:52,978 AbstractCassandraDaemon.java
(line 133) Fatal exception in thread Thread[MutationStage:8,5,main]
java.lang.OutOfMemoryError: Java heap space
at
org.apache.cassandra.io.util.RandomAccessReader.<init>(RandomAccessReader.java:75)
at
org.apache.cassandra.io.compress.CompressedRandomAccessReader.<init>(CompressedRandomAccessReader.java:62)
at
org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:44)
at
org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:39)
at
org.apache.cassandra.io.util.CompressedSegmentedFile.getSegment(CompressedSegmentedFile.java:63)
at
org.apache.cassandra.io.sstable.SSTableReader.getFileDataInput(SSTableReader.java:790)
at
org.apache.cassandra.db.columniterator.SSTableNamesIterator.<init>(SSTableNamesIterator.java:61)
at
org.apache.cassandra.db.filter.NamesQueryFilter.getSSTableColumnIterator(NamesQueryFilter.java:60)
at
org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:78)
at
org.apache.cassandra.db.CollationController.collectTimeOrderedData(CollationController.java:114)
at
org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:62)
at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1274)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1160)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1122)
at
org.apache.cassandra.db.Table.readCurrentIndexedColumns(Table.java:504)
at org.apache.cassandra.db.Table.apply(Table.java:441)
at
org.apache.cassandra.db.commitlog.CommitLog$2.runMayThrow(CommitLog.java:338)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
WARN [ScheduledTasks:1] 2011-11-27 16:36:52,902 AutoSavingCache.java (line
187) Reducing NodeIdInfo KeyCache capacity from 1 to 0 to reduce memory pressure
WARN [ScheduledTasks:1] 2011-11-27 16:36:52,982 AutoSavingCache.java (line
187) Reducing IndexInfo KeyCache capacity from 2 to 1 to reduce memory pressure
WARN [ScheduledTasks:1] 2011-11-27 16:36:52,983 AutoSavingCache.java (line
187) Reducing LocationInfo KeyCache capacity from 2 to 0 to reduce memory
pressure
WARN [ScheduledTasks:1] 2011-11-27 16:36:52,985 AutoSavingCache.java (line
187) Reducing Versions KeyCache capacity from 2 to 0 to reduce memory pressure
WARN [ScheduledTasks:1] 2011-11-27 16:36:52,985 AutoSavingCache.java (line
187) Reducing Migrations KeyCache capacity from 3 to 0 to reduce memory pressure
WARN [ScheduledTasks:1] 2011-11-27 16:36:52,986 AutoSavingCache.java (line
187) Reducing HintsColumnFamily KeyCache capacity from 5 to 0 to reduce memory
pressure
WARN [ScheduledTasks:1] 2011-11-27 16:36:52,987 AutoSavingCache.java (line
187) Reducing Schema KeyCache capacity from 3 to 1 to reduce memory pressure
WARN [ScheduledTasks:1] 2011-11-27 16:36:52,988 GCInspector.java (line 146)
Heap is 0.9563799224191398 full. You may need to reduce memtable and/or cache
sizes. Cassandra will now flush up to the two largest memtables to free up
memory. Adjust flush_largest_memtables_at threshold in cassandra.yaml if you
don't want Cassandra to do this automatically
INFO [ScheduledTasks:1] 2011-11-27 16:36:52,988 StorageService.java (line
2507) Unable to reduce heap usage since there are no dirty column families
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
