Sergey Kirillov created CASSANDRA-14253:

             Summary: MutationStage threads deadlock
                 Key: CASSANDRA-14253
             Project: Cassandra
          Issue Type: Bug
          Components: Local Write-Read Paths
         Environment: Ubuntu 16.04
            Reporter: Sergey Kirillov
         Attachments: flame.svg, flame_tn.svg


I've had a persistent problems with cluster upgraded to 3.11.2. I was 
recreating one of my MVs and suddenly nodes request latencies went crazy.

During investigation I have found that half of my nodes had stuck MutationStage 
threads. All 64 threads were Active, Pending count was continuously increasing 
while Completed stuck on one value.

After restart nodes worked a few minutes and then stuck again. Another restart, 
another few minutes of work and stuck.

In attachment you can find flame graphs for MutationStage threads and for all 
threads. It seems that all MutationStage threads were waiting for some event.


Downgrade to 3.10 solved that problem, after downgrade all nodes are 
operational and not freezing.


This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to