Ran into an issue where Cassandra Crashed when running out of heap space

2010-07-20 Thread Dathan Pattishall
 INFO [HINTED-HANDOFF-POOL:1] 2010-07-20 15:10:43,721 HintedHandOffManager.java (line 210) Finished hinted handoff of 0 rows to endpoint /10.129.28.23 ERROR [pool-1-thread-37895] 2010-07-20 15:10:51,622 CassandraDaemon.java (line 83) Uncaught exception in thread Thread[pool-1-thread-37895,5,main]

Re: Ran into an issue where Cassandra Crashed when running out of heap space

2010-07-20 Thread Peter Schuller
CassandraDaemon.java (line 83) Uncaught exception in thread Thread[pool-1-thread-37895,5,main] java.lang.OutOfMemoryError: Java heap space     at org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.java:296)     at

Re: Ran into an issue where Cassandra Crashed when running out of heap space

2010-07-20 Thread Dathan Pattishall
The storage structure is rather simple. For every 1 key there is 1 column and a timestamp for that column. ColumnFamily Name=Standard2 CompareWith=UTF8Type / We don't enable pulling a huge amount of data and all other nodes are up servicing the same request. I suspect there may be another

Re: Ran into an issue where Cassandra Crashed when running out of heap space

2010-07-20 Thread Peter Schuller
Attaching Jconsole shows that there is a growth of memory and weird spikes. Unfortunately I did not take a screen shot of the growth of the spike over time. I'll do that when it occurs again. Note that expected behavior for CMS is to have lots of small ups and downs as a result of young

Re: Ran into an issue where Cassandra Crashed when running out of heap space

2010-07-20 Thread Ryan King
On Tue, Jul 20, 2010 at 1:28 PM, Peter Schuller peter.schul...@infidyne.com wrote: Attaching Jconsole shows that there is a growth of memory and weird spikes. Unfortunately I did not take a screen shot of the growth of the spike over time. I'll do that when it occurs again. Note that expected