Re: Cassandra running High Load with no one using the cluster

2013-05-07 Thread aaron morton
Why did you increase the stack-size to 5.5 times greater than recommended? Since each threads now uses 1000KB minimum just for the stack, a large number of threads will use a large amount of memory. I'd say that is the reason you are running out of memory. Cheers - Aaron

Re: Cassandra running High Load with no one using the cluster

2013-05-06 Thread Robert Coli
On Sat, May 4, 2013 at 9:22 PM, Aiman Parvaiz ai...@grapheffect.com wrote: We are using cassandra 1.1.0 and open-6-jdk 1.1.0 has significant issues, including non-working Hinted Handoff. Also, OpenJDK is not officially supported. Upgrade to 1.1.11 and Sun JDK. =Rob

Re: Cassandra running High Load with no one using the cluster

2013-05-06 Thread Aiman Parvaiz
Correction, there was a typo in my original question, we are running cassandra 1.1.10 Thanks and sorry for the inconvenience. On May 6, 2013, at 10:23 AM, Robert Coli rc...@eventbrite.com wrote: including non-working Hinted Handoff

Re: Cassandra running High Load with no one using the cluster

2013-05-06 Thread Bryan Talbot
On Sat, May 4, 2013 at 9:22 PM, Aiman Parvaiz ai...@grapheffect.com wrote: When starting this cluster we set JVM_OPTS=$JVM_OPTS -Xss1000k Why did you increase the stack-size to 5.5 times greater than recommended? Since each threads now uses 1000KB minimum just for the stack, a large

Cassandra running High Load with no one using the cluster

2013-05-04 Thread Aiman Parvaiz
Since last night I am seeing CPU load spikes on our cassandra boxes(Occasionally load goes up to 20, its a Amazon EC2 c1.xlarge with 300 iops EBS). After digging around a little I believe its related to heap memory and flushing memtables. From logs: WARN 03:22:03,414 Heap is 0.7786981388910019