Re: cascading failures due to memory

2011-06-15 Thread AJ
Sasha, Did you ever nail down the cause of this problem? On 5/31/2011 4:01 AM, Sasha Dolgy wrote: hi everyone, the current nodes i have deployed (4) have all been working fine, with not a lot of data ... more reads than writes at the moment. as i had monitoring disabled, when one node's OS

Re: cascading failures due to memory

2011-06-15 Thread Sasha Dolgy
No. Upgraded to 0.8 and monitor the systems more. we schedule a repair every 24hrs via cron and so far no problems.. On Jun 15, 2011 5:44 PM, AJ a...@dude.podzone.net wrote: Sasha, Did you ever nail down the cause of this problem? On 5/31/2011 4:01 AM, Sasha Dolgy wrote: hi everyone, the

Re: cascading failures due to memory

2011-06-01 Thread Sasha Dolgy
is there a specific string I should be looking for in the logs that isn't super obvious to me at the moment... On Tue, May 31, 2011 at 8:21 PM, Jonathan Ellis jbel...@gmail.com wrote: The place to start is with the statistics Cassandra logs after each GC. On Tue, May 31, 2011 at 5:01 AM, Sasha

Re: cascading failures due to memory

2011-06-01 Thread Sasha Dolgy
and is there anything specific that could be causing the issue between Java SE 1.6.0_24 and 1.6.0_25 ? All nodes are _24 up to 64% memory usage today -sd On Wed, Jun 1, 2011 at 9:30 PM, Sasha Dolgy sdo...@gmail.com wrote: is there a specific string I should be looking for in the logs that

Re: cascading failures due to memory

2011-06-01 Thread Jonathan Ellis
look for GCInspector On Wed, Jun 1, 2011 at 2:30 PM, Sasha Dolgy sdo...@gmail.com wrote: is there a specific string I should be looking for in the logs that isn't super obvious to me at the moment... On Tue, May 31, 2011 at 8:21 PM, Jonathan Ellis jbel...@gmail.com wrote: The place to start

cascading failures due to memory

2011-05-31 Thread Sasha Dolgy
hi everyone, the current nodes i have deployed (4) have all been working fine, with not a lot of data ... more reads than writes at the moment. as i had monitoring disabled, when one node's OS killed the cassandra process due to out of memory problems ... that was fine. 24 hours later, another

Re: cascading failures due to memory

2011-05-31 Thread Jonathan Ellis
The place to start is with the statistics Cassandra logs after each GC. On Tue, May 31, 2011 at 5:01 AM, Sasha Dolgy sdo...@gmail.com wrote: hi everyone, the current nodes i have deployed (4) have all been working fine, with not a lot of data ... more reads than writes at the moment.  as i