XBaith commented on issue #2623:
URL: https://github.com/apache/amoro/issues/2623#issuecomment-2071493190
There still seems to be a deadlock risk when multiple threads cancel the
same task at the same time.
```
Found one Java-level deadlock:
=============================
"thrift-server-optimize-manager-35922":
waiting for ownable synchronizer 0x00000005d6cac510, (a
java.util.concurrent.locks.ReentrantLock$NonfairSync),
which is held by "thrift-server-optimize-manager-35890"
"thrift-server-optimize-manager-35890":
waiting for ownable synchronizer 0x00000005d6c886e8, (a
java.util.concurrent.locks.ReentrantLock$NonfairSync),
which is held by "thrift-server-optimize-manager-35922"
Java stack information for the threads listed above:
===================================================
"thrift-server-optimize-manager-35922":
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000005d6cac510> (a
java.util.concurrent.locks.ReentrantLock$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
at
java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)
at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)
at
com.netease.arctic.server.persistence.StatedPersistentBase.invokeConsistency(StatedPersistentBase.java:49)
at
com.netease.arctic.server.optimizing.TaskRuntime.tryCanceling(TaskRuntime.java:146)
at
com.netease.arctic.server.optimizing.OptimizingQueue$TableOptimizingProcess$$Lambda$3583/646015585.accept(Unknown
Source)
at java.util.HashMap$Values.forEach(HashMap.java:982)
at
com.netease.arctic.server.optimizing.OptimizingQueue$TableOptimizingProcess.cancelTasks(OptimizingQueue.java:652)
at
com.netease.arctic.server.optimizing.OptimizingQueue$TableOptimizingProcess.acceptResult(OptimizingQueue.java:473)
at
com.netease.arctic.server.optimizing.TaskRuntime.lambda$complete$0(TaskRuntime.java:103)
at
com.netease.arctic.server.optimizing.TaskRuntime$$Lambda$3511/1325215119.run(Unknown
Source)
at
com.netease.arctic.server.persistence.PersistentBase$$Lambda$1082/209258523.accept(Unknown
Source)
at
java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
at
java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:647)
at
com.netease.arctic.server.persistence.PersistentBase.doAsTransaction(PersistentBase.java:61)
at
com.netease.arctic.server.persistence.StatedPersistentBase.invokeConsistency(StatedPersistentBase.java:52)
at
com.netease.arctic.server.optimizing.TaskRuntime.complete(TaskRuntime.java:76)
at
com.netease.arctic.server.DefaultOptimizingService.completeTask(DefaultOptimizingService.java:235)
at sun.reflect.GeneratedMethodAccessor239.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.netease.arctic.server.utils.ThriftServiceProxy.invoke(ThriftServiceProxy.java:56)
at com.sun.proxy.$Proxy48.completeTask(Unknown Source)
at
com.netease.arctic.api.OptimizingService$Processor$completeTask.getResult(OptimizingService.java:583)
at
com.netease.arctic.api.OptimizingService$Processor$completeTask.getResult(OptimizingService.java:562)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38)
at
org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:138)
at
org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:524)
at org.apache.thrift.server.Invocation.run(Invocation.java:18)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
"thrift-server-optimize-manager-35890":
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000005d6c886e8> (a
java.util.concurrent.locks.ReentrantLock$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
at
java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)
at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)
at
com.netease.arctic.server.persistence.StatedPersistentBase.invokeConsistency(StatedPersistentBase.java:49)
at
com.netease.arctic.server.optimizing.TaskRuntime.tryCanceling(TaskRuntime.java:146)
at
com.netease.arctic.server.optimizing.OptimizingQueue$TableOptimizingProcess$$Lambda$3583/646015585.accept(Unknown
Source)
at java.util.HashMap$Values.forEach(HashMap.java:982)
at
com.netease.arctic.server.optimizing.OptimizingQueue$TableOptimizingProcess.cancelTasks(OptimizingQueue.java:652)
at
com.netease.arctic.server.optimizing.OptimizingQueue$TableOptimizingProcess.acceptResult(OptimizingQueue.java:473)
at
com.netease.arctic.server.optimizing.TaskRuntime.lambda$complete$0(TaskRuntime.java:103)
at
com.netease.arctic.server.optimizing.TaskRuntime$$Lambda$3511/1325215119.run(Unknown
Source)
at
com.netease.arctic.server.persistence.PersistentBase$$Lambda$1082/209258523.accept(Unknown
Source)
at
java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
at
java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:647)
at
com.netease.arctic.server.persistence.PersistentBase.doAsTransaction(PersistentBase.java:61)
at
com.netease.arctic.server.persistence.StatedPersistentBase.invokeConsistency(StatedPersistentBase.java:52)
at
com.netease.arctic.server.optimizing.TaskRuntime.complete(TaskRuntime.java:76)
at
com.netease.arctic.server.DefaultOptimizingService.completeTask(DefaultOptimizingService.java:235)
at sun.reflect.GeneratedMethodAccessor239.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.netease.arctic.server.utils.ThriftServiceProxy.invoke(ThriftServiceProxy.java:56)
at com.sun.proxy.$Proxy48.completeTask(Unknown Source)
at
com.netease.arctic.api.OptimizingService$Processor$completeTask.getResult(OptimizingService.java:583)
at
com.netease.arctic.api.OptimizingService$Processor$completeTask.getResult(OptimizingService.java:562)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38)
at
org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:138)
at
org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:524)
at org.apache.thrift.server.Invocation.run(Invocation.java:18)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
