Fuud created CASSANDRA-13652: -------------------------------- Summary: Deadlock in AbstractCompactionManager Key: CASSANDRA-13652 URL: https://issues.apache.org/jira/browse/CASSANDRA-13652 Project: Cassandra Issue Type: Bug Components: Core Reporter: Fuud
AbstractCommitLogManager uses LockSupport.(un)park incorreclty. It invokes unpark without checking if manager thread was parked in approriate place. For example, logging frameworks uses queues and queues uses ReadWriteLock's that uses LockSupport. Therefore AbstractCommitLogManager.wakeManager can wake thread inside Lock and manager thread will sleep forever at park() method (because unpark permit was already consumed inside lock). For examle stack traces: {code} "MigrationStage:1" id=412 state=WAITING at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at org.apache.cassandra.utils.concurrent.WaitQueue$AbstractSignal.awaitUninterruptibly(WaitQueue.java:279) at org.apache.cassandra.db.commitlog.AbstractCommitLogSegmentManager.awaitAvailableSegment(AbstractCommitLogSegmentManager.java:263) at org.apache.cassandra.db.commitlog.AbstractCommitLogSegmentManager.advanceAllocatingFrom(AbstractCommitLogSegmentManager.java:237) at org.apache.cassandra.db.commitlog.AbstractCommitLogSegmentManager.forceRecycleAll(AbstractCommitLogSegmentManager.java:279) at org.apache.cassandra.db.commitlog.CommitLog.forceRecycleAllSegments(CommitLog.java:210) at org.apache.cassandra.config.Schema.dropView(Schema.java:708) at org.apache.cassandra.schema.SchemaKeyspace.lambda$updateKeyspace$23(SchemaKeyspace.java:1361) at org.apache.cassandra.schema.SchemaKeyspace$$Lambda$382/1123232162.accept(Unknown Source) at java.util.LinkedHashMap$LinkedValues.forEach(LinkedHashMap.java:608) at java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1080) at org.apache.cassandra.schema.SchemaKeyspace.updateKeyspace(SchemaKeyspace.java:1361) at org.apache.cassandra.schema.SchemaKeyspace.mergeSchema(SchemaKeyspace.java:1332) at org.apache.cassandra.schema.SchemaKeyspace.mergeSchemaAndAnnounceVersion(SchemaKeyspace.java:1282) - locked java.lang.Class@cc38904 at org.apache.cassandra.db.DefinitionsUpdateVerbHandler$1.runMayThrow(DefinitionsUpdateVerbHandler.java:51) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$LocalSessionWrapper.run(DebuggableThreadPoolExecutor.java:322) at com.ringcentral.concurrent.executors.MonitoredRunnable.run(MonitoredRunnable.java:36) at MON_R_MigrationStage.run(NamedRunnableFactory.java:67) at com.ringcentral.concurrent.executors.MonitoredThreadPoolExecutor$MdcAwareRunnable.run(MonitoredThreadPoolExecutor.java:114) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79) at org.apache.cassandra.concurrent.NamedThreadFactory$$Lambda$61/1733339045.run(Unknown Source) at java.lang.Thread.run(Thread.java:745) "COMMIT-LOG-ALLOCATOR:1" id=80 state=WAITING at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at org.apache.cassandra.db.commitlog.AbstractCommitLogSegmentManager$1.runMayThrow(AbstractCommitLogSegmentManager.java:128) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) at org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79) at org.apache.cassandra.concurrent.NamedThreadFactory$$Lambda$61/1733339045.run(Unknown Source) at java.lang.Thread.run(Thread.java:745) {code} Solution is to use Semaphore instead of low-level LockSupport. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org