Clint, did you find anything? I just noticed it happens to us too on only one node in our CI cluster. I don't think there is a special usage before it happens... The last line in the log before the shutdown lines in at least an hour before.. We're using C* 2.0.9.
On Thu, Aug 7, 2014 at 12:49 AM, Clint Kelly <clint.ke...@gmail.com> wrote: > Hi Rob, > > Thanks for the clarification; this is really useful. I'll run some > experiments to see if the problem is a JVM OOM on our build machine. > > Best regards, > Clint > > On Wed, Aug 6, 2014 at 1:14 PM, Robert Coli <rc...@eventbrite.com> wrote: > > On Wed, Aug 6, 2014 at 1:12 PM, Robert Coli <rc...@eventbrite.com> > wrote: > >> > >> On Wed, Aug 6, 2014 at 1:11 AM, Duncan Sands <duncan.sa...@gmail.com> > >> wrote: > >>> > >>> this doesn't look like an OOM to me. If the kernel OOM kills Cassandra > >>> then Cassandra instantly vaporizes, and there will be nothing in the > >>> Cassandra logs (you will find information about the OOM in the system > logs > >>> though, eg in dmesg). In the log snippet above you see an orderly > shutdown, > >>> this is completely different to the instant OOM kill. > >> > >> > >> Not really. > >> > >> https://issues.apache.org/jira/browse/CASSANDRA-7507 > > > > > > To be clear, there's two different OOMs here, I am talking about the JVM > > OOM, not system level. As CASSANDRA-7507 indicates, JVM OOM does not > > necessarily result in the cassandra process dying, and can in fact > trigger > > clean shutdown. > > > > System level OOM will in fact send the equivalent of KILL, which will not > > trigger the clean shutdown hook in Cassandra. > > > > =Rob > -- Or Sher
