[
https://issues.apache.org/jira/browse/CASSANDRA-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12986807#action_12986807
]
David King commented on CASSANDRA-2058:
---------------------------------------
Also since then I've had notably worse performance, reading is maybe 30% slower
than before.
My next step will be to hope that the jstacks in that log are the same as the
ones causing the largest outages and to disable to dynamic snitch (as much as
i'd like to get 100% reproduction, I'd also rather not take my site down) to
see if that resolves the problem. If it doesn't, then I'll turn it back on and
revert to 0.6.8 to see if that does it
> Nodes periodically spike in load
> --------------------------------
>
> Key: CASSANDRA-2058
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2058
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.6.10
> Reporter: David King
> Attachments: cassandra.pmc01.log.bz2
>
>
> (Filing as a placeholder bug as I gather information.)
> At ~10p 24 Jan, I upgraded our 20-node cluster from 0.6.8->0.6.10, turned on
> the DES, and moved some CFs from one KS into another (drain whole cluster,
> take it down, move files, change schema, put it back up). Since then, I've
> had four storms whereby a node's load will shoot to 700+ (400% CPU on a 4-cpu
> machine) and become totally unresponsive. After a moment or two like that,
> its neighbour dies too, and the failure cascades around the ring.
> Unfortunately because of the high load I'm not able to get into the machine
> to pull a thread dump to see wtf it's doing as it happens.
> I've also had an issue where a single node spikes up to high load, but
> recovers. This may or may not be the same issue from which the nodes don't
> recover as above, but both are new behaviour
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
