Hello,
I am currently using Cassandra-2.0.0 on OpenSuse for storing
netflow packets that are seen on my ethernet interface. I deliberately
tried to test Cassandra with heavy data and it ran fine for about 30 mins
after which it crashed with OutOfMemory error. I set up a two-node cluster
to which this data is getting stored with replication_factor 1. Now,
Cassandra is not even starting up. The log is given below for your
reference.
Can I solve
this problem by tweaking JVM OPTS? If yes, which all and how? How can I be
sure that it is not someother issue like corrupted commit log headers, etc.
so as to prevent these errors in the future? I am on a 32-bit OpenSuse i5
machine with 4G RAM.
Here is the output when I try to start Cassandra:-
linux-0cpn:~/bnsk/
experimentation/apache-cassandra-2.0.0/bin # ./cassandra -f &
[1] 984
linux-0cpn:~/bnsk/experimentation/apache-cassandra-2.0.0/bin #
./../conf/cassandra-env.sh: line 137: elseError:: command not found
xss = -ea -javaagent:./../lib/jamm-0.2.5.jar -XX:+UseThreadPriorities
-XX:ThreadPriorityPolicy=42 -Xms1024M -Xmx1024M -Xmn256M
-XX:+HeapDumpOnOutOfMemoryError -Xss256k
INFO 03:46:43,178 Logging initialized
INFO 03:46:43,188 32bit JVM detected. It is recommended to run Cassandra
on a 64bit JVM for better performance.
INFO 03:46:43,189 JVM vendor/version: Java HotSpot(TM) Server VM/1.7.0_45
INFO 03:46:43,189 Heap size: 1046937600/1046937600
INFO 03:46:43,189 Classpath:
./../conf:./../build/classes/main:./../build/classes/thrift:./../lib/antlr-3.2.jar:./../lib/apache-cassandra-2.0.0.jar:./../lib/apache-cassandra-clientutil-2.0.0.jar:./../lib/apache-cassandra-thrift-2.0.0.jar:./../lib/commons-cli-1.1.jar:./../lib/commons-codec-1.2.jar:./../lib/commons-lang-2.6.jar:./../lib/compress-lzf-0.8.4.jar:./../lib/concurrentlinkedhashmap-lru-1.3.jar:./../lib/disruptor-3.0.1.jar:./../lib/guava-13.0.1.jar:./../lib/high-scale-lib-1.1.2.jar:./../lib/jackson-core-asl-1.9.2.jar:./../lib/jackson-mapper-asl-1.9.2.jar:./../lib/jamm-0.2.5.jar:./../lib/jbcrypt-0.3m.jar:./../lib/jline-1.0.jar:./../lib/json-simple-1.1.jar:./../lib/libthrift-0.9.0.jar:./../lib/log4j-1.2.16.jar:./../lib/lz4-1.1.0.jar:./../lib/metrics-core-2.0.3.jar:./../lib/netty-3.5.9.Final.jar:./../lib/servlet-api-2.5-20081211.jar:./../lib/slf4j-api-1.7.2.jar:./../lib/slf4j-log4j12-1.7.2.jar:./../lib/snakeyaml-1.11.jar:./../lib/snappy-java-1.0.5.jar:./../lib/snaptree-0.1.jar:./../lib/thrift-server-0.3.0.jar:./../lib/jamm-0.2.5.jar
INFO 03:46:43,191 JNA not found. Native methods will be disabled.
INFO 03:46:43,199 Loading settings from
file:/root/bnsk/experimentation/apache-cassandra-2.0.0/conf/cassandra.yaml
INFO 03:46:43,418 Data files directories: [/var/lib/cassandra/data]
INFO 03:46:43,418 Commit log directory: /var/lib/cassandra/commitlog
INFO 03:46:43,418 DiskAccessMode 'auto' determined to be standard,
indexAccessMode is standard
INFO 03:46:43,418 disk_failure_policy is stop
INFO 03:46:43,422 Global memtable threshold is enabled at 332MB
INFO 03:46:43,510 Not using multi-threaded compaction
INFO 03:46:43,660 Initializing key cache with capacity of 49 MBs.
INFO 03:46:43,667 Scheduling key cache save to each 14400 seconds (going
to save all keys).
INFO 03:46:43,668 Initializing row cache with capacity of 0 MBs
INFO 03:46:43,674 Scheduling row cache save to each 0 seconds (going to
save all keys).
INFO 03:46:43,755 Initializing system.schema_triggers
INFO 03:46:43,768 Opening
/var/lib/cassandra/data/system/schema_triggers/system-schema_triggers-ja-104
(57 bytes)
INFO 03:46:43,768 Opening
/var/lib/cassandra/data/system/schema_triggers/system-schema_triggers-ja-105
(57 bytes)
INFO 03:46:43,769 Opening
/var/lib/cassandra/data/system/schema_triggers/system-schema_triggers-ja-103
(57 bytes)
INFO 03:46:43,790 reading saved cache
/var/lib/cassandra/saved_caches/system-schema_triggers-KeyCache-b.db
INFO 03:46:43,798 Initializing system.batchlog
INFO 03:46:43,800 Initializing system.peer_events
INFO 03:46:43,804 Initializing system.compactions_in_progress
INFO 03:46:43,805 Opening
/var/lib/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-ja-22
(177 bytes)
INFO 03:46:43,805 Opening
/var/lib/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-ja-23
(42 bytes)
INFO 03:46:43,809 Initializing system.hints
INFO 03:46:43,812 Initializing system.schema_keyspaces
INFO 03:46:43,815 Opening
/var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-ja-493
(554 bytes)
INFO 03:46:43,816 Opening
/var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-ja-494
(473 bytes)
INFO 03:46:43,818 reading saved cache
/var/lib/cassandra/saved_caches/system-schema_keyspaces-KeyCache-b.db
INFO 03:46:43,821 Initializing system.range_xfers
INFO 03:46:43,824 Initializing system.schema_columnfamilies
INFO 03:46:43,826 Opening
/var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-ja-535
(77 bytes)
INFO 03:46:43,827 Opening
/var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-ja-534
(75 bytes)
INFO 03:46:43,833 Opening
/var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-ja-533
(8589 bytes)
INFO 03:46:43,841 reading saved cache
/var/lib/cassandra/saved_caches/system-schema_columnfamilies-KeyCache-b.db
java.lang.OutOfMemoryError: Java heap space
Dumping heap to java_pid984.hprof ...
Heap dump file created [15493326 bytes in 0.068 secs]
ERROR 03:46:43,995 Exception encountered during startup
java.lang.OutOfMemoryError: Java heap space
at
org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:394)
at
org.apache.cassandra.utils.ByteBufferUtil.readWithLength(ByteBufferUtil.java:355)
at
org.apache.cassandra.service.CacheService$KeyCacheSerializer.deserialize(CacheService.java:337)
at
org.apache.cassandra.cache.AutoSavingCache.loadSaved(AutoSavingCache.java:147)
at
org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:267)
at
org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:411)
at
org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:383)
at org.apache.cassandra.db.Keyspace.initCf(Keyspace.java:314)
at org.apache.cassandra.db.Keyspace.<init>(Keyspace.java:268)
at org.apache.cassandra.db.Keyspace.open(Keyspace.java:110)
at org.apache.cassandra.db.Keyspace.open(Keyspace.java:88)
at
org.apache.cassandra.db.SystemKeyspace.checkHealth(SystemKeyspace.java:447)
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:227)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:443)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:486)
java.lang.OutOfMemoryError: Java heap space
at
org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:394)
at
org.apache.cassandra.utils.ByteBufferUtil.readWithLength(ByteBufferUtil.java:355)
at
org.apache.cassandra.service.CacheService$KeyCacheSerializer.deserialize(CacheService.java:337)
at
org.apache.cassandra.cache.AutoSavingCache.loadSaved(AutoSavingCache.java:147)
at
org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:267)
at
org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:411)
at
org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:383)
at org.apache.cassandra.db.Keyspace.initCf(Keyspace.java:314)
at org.apache.cassandra.db.Keyspace.<init>(Keyspace.java:268)
at org.apache.cassandra.db.Keyspace.open(Keyspace.java:110)
at org.apache.cassandra.db.Keyspace.open(Keyspace.java:88)
at
org.apache.cassandra.db.SystemKeyspace.checkHealth(SystemKeyspace.java:447)
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:227)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:443)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:486)
Exception encountered during startup: Java heap space
Thanks in advance...
--
Regards,
BNSK
*.*
