eolivelli commented on pull request #9764:
URL: https://github.com/apache/pulsar/pull/9764#issuecomment-789618833
I see now, this is a case of deadlock, we are issuing a blocking call to
ZooKeeper into a forEach, and the ZooKeeper library handles every operation in
only one single thread.
So the chain is pretty hard do understand
```
"pulsar-backlog-quota-checker-146-1" prio=5 tid=258 TIMED_WAITING
java.lang.Thread.State: TIMED_WAITING
at sun.misc.Unsafe.park(Native Method)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1709)
at
java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
at
java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1788)
at
java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1928)
at
org.apache.pulsar.zookeeper.ZooKeeperDataCache.get(ZooKeeperDataCache.java:97)
at
org.apache.pulsar.broker.service.BacklogQuotaManager.getBacklogQuota(BacklogQuotaManager.java:66)
at
org.apache.pulsar.broker.service.BacklogQuotaManager.getBacklogQuota(BacklogQuotaManager.java:78)
at
org.apache.pulsar.broker.service.BacklogQuotaManager.getBacklogQuotaLimit(BacklogQuotaManager.java:94)
at
org.apache.pulsar.broker.service.persistent.PersistentTopic.isBacklogExceeded(PersistentTopic.java:2227)
at
org.apache.pulsar.broker.service.BrokerService.lambda$monitorBacklogQuota$53(BrokerService.java:1502)
at
org.apache.pulsar.broker.service.BrokerService$$Lambda$453/1035521358.accept(Unknown
Source)
at
org.apache.pulsar.broker.service.BrokerService$$Lambda$858/1296923805.accept(Unknown
Source)
at java.util.Optional.ifPresent(Optional.java:159)
at
org.apache.pulsar.broker.service.BrokerService.lambda$forEachTopic$52(BrokerService.java:1490)
at
org.apache.pulsar.broker.service.BrokerService$$Lambda$454/965168673.accept(Unknown
Source)
at
org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap$Section.forEach(ConcurrentOpenHashMap.java:387)
at
org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap.forEach(ConcurrentOpenHashMap.java:159)
at
org.apache.pulsar.broker.service.BrokerService.forEachTopic(BrokerService.java:1488)
at
org.apache.pulsar.broker.service.BrokerService.monitorBacklogQuota(BrokerService.java:1499)
at
org.apache.pulsar.broker.service.BrokerService$$Lambda$280/2017773859.run(Unknown
Source)
at
org.apache.bookkeeper.mledger.util.SafeRun$1.safeRun(SafeRun.java:32)
at
org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
