Re: Kernel keeps killing cassandra process - OOM

2014-03-27 Thread prem yadav
Hi all, thanks for the help. the issue happened again. I looked into Romain's suggestion and yes, it was VMware balloning. So, i will not be updating the jira ticket as I am not sure if the fix actually helped or not. Thanks, Prem On Wed, Mar 26, 2014 at 6:54 PM, Robert Coli

Re: Kernel keeps killing cassandra process - OOM

2014-03-26 Thread prem yadav
Thanks Robert. That seems to be the issue. however the fix mentioned there doesn't work. I downgraded Java to jdk6_37 and that seems to have done the trick. Thanks for pointing me to that Jira ticket. On Mon, Mar 24, 2014 at 6:48 PM, Robert Coli rc...@eventbrite.com wrote: On Mon, Mar 24, 2014

Re: Kernel keeps killing cassandra process - OOM

2014-03-26 Thread Robert Coli
On Wed, Mar 26, 2014 at 8:35 AM, prem yadav ipremya...@gmail.com wrote: Thanks Robert. That seems to be the issue. however the fix mentioned there doesn't work. I downgraded Java to jdk6_37 and that seems to have done the trick. Thanks for pointing me to that Jira ticket. If the workaround

Re: Kernel keeps killing cassandra process - OOM

2014-03-24 Thread Romain HARDOUIN
: Kernel keeps killing cassandra process - OOM You might want to look at: http://www.opensourceconnections.com/2013/08/31/building-the- perfect-cassandra-test-environment/

Re: Kernel keeps killing cassandra process - OOM

2014-03-24 Thread prem yadav
, Michael michael.la...@nytimes.com A : user@cassandra.apache.org, Date : 22/03/2014 22:26 Objet : Re: Kernel keeps killing cassandra process - OOM You might want to look at: http://www.opensourceconnections.com/2013/08/31/building-the- perfect-cassandra-test-environment/

Re: Kernel keeps killing cassandra process - OOM

2014-03-24 Thread Romain HARDOUIN
:31 : De : prem yadav ipremya...@gmail.com A : user@cassandra.apache.org, Date : 24/03/2014 12:12 Objet : Re: Kernel keeps killing cassandra process - OOM the nodes die without being under any load. Completely idle. And 4 GB system memory is not low. or is it? I have tried tweaking

Re: Kernel keeps killing cassandra process - OOM

2014-03-24 Thread Robert Coli
On Mon, Mar 24, 2014 at 4:11 AM, prem yadav ipremya...@gmail.com wrote: the nodes die *without * being under any load. Completely idle. https://issues.apache.org/jira/browse/CASSANDRA-6541 ? =Rob

Kernel keeps killing cassandra process - OOM

2014-03-22 Thread prem yadav
Hi, I have a 3 node cassandra test cluster. The nodes have 4 GB total memory/2 cores. Cassndra run with all default settings. But, the cassandra process keeps getting killed due to OOM. Cassandra version in use is 1.1.9. here are the settings in use: compaction_throughput_mb_per_sec: 16

Re: Kernel keeps killing cassandra process - OOM

2014-03-22 Thread Michael Dykman
What JVM are you running on? What, if any, memory constraints are you passing to the process? On Mar 22, 2014 10:48 AM, prem yadav ipremya...@gmail.com wrote: Hi, I have a 3 node cassandra test cluster. The nodes have 4 GB total memory/2 cores. Cassndra run with all default settings. But,

Re: Kernel keeps killing cassandra process - OOM

2014-03-22 Thread Robert Coli
On Sat, Mar 22, 2014 at 7:48 AM, prem yadav ipremya...@gmail.com wrote: But, the cassandra process keeps getting killed due to OOM. Cassandra version in use is 1.1.9. Try using 1.2.15, instead? =Rob

Re: Kernel keeps killing cassandra process - OOM

2014-03-22 Thread prem yadav
Its Oracle jdk 1.6. Robert, any fix that you know of which went into 1.2.15 for this particular issue? On Sat, Mar 22, 2014 at 4:50 PM, Robert Coli rc...@eventbrite.com wrote: On Sat, Mar 22, 2014 at 7:48 AM, prem yadav ipremya...@gmail.com wrote: But, the cassandra process keeps getting

Re: Kernel keeps killing cassandra process - OOM

2014-03-22 Thread prem yadav
Michael, no memory constraints. System memory is 4 GB and Cassandra run on default. On Sat, Mar 22, 2014 at 5:32 PM, prem yadav ipremya...@gmail.com wrote: Its Oracle jdk 1.6. Robert, any fix that you know of which went into 1.2.15 for this particular issue? On Sat, Mar 22, 2014 at 4:50

Re: Kernel keeps killing cassandra process - OOM

2014-03-22 Thread Laing, Michael
I ran into the same problem some time ago. Upgrading to Cassandra 2, jdk 1.7, and default parameters fixed it. I think the jdk change was the key for my similarly small memory cluster. ml On Sat, Mar 22, 2014 at 1:36 PM, prem yadav ipremya...@gmail.com wrote: Michael, no memory

Re: Kernel keeps killing cassandra process - OOM

2014-03-22 Thread prem yadav
Upgrading is not possible right now. Any other suggestions guys? I have already tried reducing the number of rpc threads. Also tried reducing the linux kernel overcommit. On Sat, Mar 22, 2014 at 5:44 PM, Laing, Michael michael.la...@nytimes.comwrote: I ran into the same problem some time ago.

Re: Kernel keeps killing cassandra process - OOM

2014-03-22 Thread Brian Flad
Is there anything else running on the boxes? Can you show us the output of ps aux for the Cassandra process so we can see the -xmx, etc? JDK 7 may help, even if you cannot upgrade Cassandra yet (which I would really recommend since it moves items off-heap). On Sat, Mar 22, 2014 at 4:01 PM, prem

Re: Kernel keeps killing cassandra process - OOM

2014-03-22 Thread prem yadav
The output of ps waux . Also there is no load on cluster. None USER PID %CPU %MEMVSZ RSS TTY STAT START TIME COMMAND root 1 0.0 0.0 19224 1076 ?Ss Mar19 0:01 /sbin/init root 2 0.0 0.0 0 0 ?SMar19 0:00 [kthreadd] root

Re: Kernel keeps killing cassandra process - OOM

2014-03-22 Thread prem yadav
Also we use datastax. The version cassandra-1.1.9 doesn't work with java 7 On Sat, Mar 22, 2014 at 9:09 PM, prem yadav ipremya...@gmail.com wrote: The output of ps waux . Also there is no load on cluster. None USER PID %CPU %MEMVSZ RSS TTY STAT START TIME COMMAND root

Re: Kernel keeps killing cassandra process - OOM

2014-03-22 Thread Laing, Michael
You might want to look at: http://www.opensourceconnections.com/2013/08/31/building-the-perfect-cassandra-test-environment/ On Sat, Mar 22, 2014 at 4:38 PM, Brian Flad bflad...@gmail.com wrote: Is there anything else running on the boxes? Can you show us the output of ps aux for the