jkrauss82 commented on PR #9153:
URL: https://github.com/apache/storm/pull/9153#issuecomment-6054380296
We had another supervisor failure / crash due to an - according to source
code - intentionally thrown RuntimeException in
`storm-server/src/main/java/org/apache/storm/daemon/supervisor/ReadClusterState.java`.
The fix for this would be:
```diff
diff --git
a/storm-server/src/main/java/org/apache/storm/daemon/supervisor/ReadClusterState.java
b/storm-server/src/main/java/org/apache/storm/daemon/supervisor/ReadClusterState.java
index 0186230d0..eb1fbaf41 100644
---
a/storm-server/src/main/java/org/apache/storm/daemon/supervisor/ReadClusterState.java
+++
b/storm-server/src/main/java/org/apache/storm/daemon/supervisor/ReadClusterState.java
@@ -186,8 +186,7 @@ public class ReadClusterState implements Runnable,
AutoCloseable {
}
} catch (Exception e) {
- LOG.error("Failed to Sync Supervisor", e);
- throw new RuntimeException(e);
+ LOG.error("Failed to Sync Supervisor, will retry next cycle",
e);
}
}
```
Since the error is logged at this location, I wonder why we re-throw in the
first place? What is the rationale to let supervisor be killed instead of
waiting out the transient state of a zookeeper connection loss? The reason for
the zookeeper downtime seems to be related to multiple topologies (40+) getting
restarted during a short timespan in the background, which seems to overwhelm
the zookeeper or nimbus, but this is nothing supervisor can do anything about
by crashing.
UNLESS there is some sync chain or so being triggered on a supervisor start
up which I am not aware about, which could help to resolve the situation. But
then I would think this sync chain should be triggered from within the running
supervisor process, not by crashing and starting again, which has several
drawbacks from a cluster point of view, e.g. the supervisor getting blacklisted.
If possible, I would like to add this fix to this PR, too, please advise.
Log of the crash, it was stalled a while by the new
`ConnectionAwareRetryPolicy` but eventually time ran out:
```
o.a.s.u.ConnectionAwareRetryPolicy HBTimer [WARN] ZK connection not
re-established within 20000ms, abandoning retry
o.a.s.d.s.t.SupervisorHeartbeat HBTimer [WARN] Supervisor heartbeat failed,
will retry next cycle
java.lang.RuntimeException:
org.apache.storm.shade.org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss for /supervisors
at org.apache.storm.utils.Utils.wrapInRuntime(Utils.java:504)
at
org.apache.storm.zookeeper.ClientZookeeper.existsNode(ClientZookeeper.java:147)
at
org.apache.storm.zookeeper.ClientZookeeper.mkdirsImpl(ClientZookeeper.java:289)
at
org.apache.storm.zookeeper.ClientZookeeper.mkdirs(ClientZookeeper.java:70)
at
org.apache.storm.cluster.ZKStateStorage.set_ephemeral_node(ZKStateStorage.java:125)
at
org.apache.storm.cluster.PaceMakerStateStorage.set_ephemeral_node(PaceMakerStateStorage.java:87)
at
org.apache.storm.cluster.StormClusterStateImpl.supervisorHeartbeat(StormClusterStateImpl.java:532)
at
org.apache.storm.daemon.supervisor.timer.SupervisorHeartbeat.run(SupervisorHeartbeat.java:166)
at org.apache.storm.StormTimer$1.run(StormTimer.java:110)
at org.apache.storm.StormTimer$StormTimerTask.run(StormTimer.java:226)
Caused by:
org.apache.storm.shade.org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss for /supervisors
at
org.apache.storm.shade.org.apache.zookeeper.KeeperException.create(KeeperException.java:101)
at
org.apache.storm.shade.org.apache.zookeeper.KeeperException.create(KeeperException.java:53)
at
org.apache.storm.shade.org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1867)
at
org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl$3.call(ExistsBuilderImpl.java:247)
at
org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl$3.call(ExistsBuilderImpl.java:240)
at
org.apache.storm.shade.org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:88)
at
org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForegroundStandard(ExistsBuilderImpl.java:240)
at
org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForeground(ExistsBuilderImpl.java:235)
at
org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:202)
at
org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:35)
at
org.apache.storm.zookeeper.ClientZookeeper.existsNode(ClientZookeeper.java:144)
... 8 more
o.a.s.s.o.a.z.ClientCnxn main-SendThread() [WARN] Session 0x0 for server
nimbus-nbg1-2/10.110.3.112:2181, Closing socket connection. Attempting
reconnect except it is a SessionExpiredException or SessionTimeoutException.
java.io.IOException: Connection has already been closed and reconnection is
not allowed
at
org.apache.storm.shade.org.apache.zookeeper.ClientCnxn$SendThread.changeZkState(ClientCnxn.java:991)
at
org.apache.storm.shade.org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1142)
at
org.apache.storm.shade.org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1200)
o.a.s.s.o.a.c.ConnectionState main-EventThread [WARN] Session expired event
received
o.a.s.u.ConnectionAwareRetryPolicy Thread-3 [WARN] Interrupted while waiting
for ZK reconnection
java.lang.InterruptedException
at java.base/java.lang.Object.wait0(Native Method)
at java.base/java.lang.Object.wait(Object.java:389)
at
org.apache.storm.shade.org.apache.curator.framework.state.ConnectionStateManager.blockUntilConnected(ConnectionStateManager.java:214)
at
org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.blockUntilConnected(CuratorFrameworkImpl.java:275)
at
org.apache.storm.utils.ConnectionAwareRetryPolicy.allowRetry(ConnectionAwareRetryPolicy.java:103)
at
org.apache.storm.shade.org.apache.curator.RetryLoopImpl.takeException(RetryLoopImpl.java:67)
at
org.apache.storm.shade.org.apache.curator.connection.ThreadLocalRetryLoop$WrappedRetryLoop.takeException(ThreadLocalRetryLoop.java:105)
at
org.apache.storm.shade.org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:92)
at
org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForegroundStandard(ExistsBuilderImpl.java:240)
at
org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForeground(ExistsBuilderImpl.java:235)
at
org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:202)
at
org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:35)
at
org.apache.storm.zookeeper.ClientZookeeper.existsNode(ClientZookeeper.java:144)
at
org.apache.storm.cluster.ZKStateStorage.node_exists(ZKStateStorage.java:150)
at
org.apache.storm.cluster.PaceMakerStateStorage.node_exists(PaceMakerStateStorage.java:97)
at
org.apache.storm.cluster.StormClusterStateImpl.getTopologyProfileRequests(StormClusterStateImpl.java:366)
at
org.apache.storm.daemon.supervisor.ReadClusterState.getProfileActions(ReadClusterState.java:202)
at
org.apache.storm.daemon.supervisor.ReadClusterState.run(ReadClusterState.java:143)
at
org.apache.storm.daemon.supervisor.timer.SynchronizeAssignments.run(SynchronizeAssignments.java:86)
at org.apache.storm.event.EventManagerImp$1.run(EventManagerImp.java:49)
o.a.s.d.s.ReadClusterState Thread-3 [ERROR] Failed to Sync Supervisor
java.lang.RuntimeException:
org.apache.storm.shade.org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for
/profilerconfigs/finanzen_net_forum-213-1790677956
at org.apache.storm.utils.Utils.wrapInRuntime(Utils.java:504)
at
org.apache.storm.zookeeper.ClientZookeeper.existsNode(ClientZookeeper.java:147)
at
org.apache.storm.cluster.ZKStateStorage.node_exists(ZKStateStorage.java:150)
at
org.apache.storm.cluster.PaceMakerStateStorage.node_exists(PaceMakerStateStorage.java:97)
at
org.apache.storm.cluster.StormClusterStateImpl.getTopologyProfileRequests(StormClusterStateImpl.java:366)
at
org.apache.storm.daemon.supervisor.ReadClusterState.getProfileActions(ReadClusterState.java:202)
at
org.apache.storm.daemon.supervisor.ReadClusterState.run(ReadClusterState.java:143)
at
org.apache.storm.daemon.supervisor.timer.SynchronizeAssignments.run(SynchronizeAssignments.java:86)
at org.apache.storm.event.EventManagerImp$1.run(EventManagerImp.java:49)
Caused by:
org.apache.storm.shade.org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for
/profilerconfigs/finanzen_net_forum-213-1790677956
at
org.apache.storm.shade.org.apache.zookeeper.KeeperException.create(KeeperException.java:133)
at
org.apache.storm.shade.org.apache.zookeeper.KeeperException.create(KeeperException.java:53)
at
org.apache.storm.shade.org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1867)
at
org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl$3.call(ExistsBuilderImpl.java:247)
at
org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl$3.call(ExistsBuilderImpl.java:240)
at
org.apache.storm.shade.org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:88)
at
org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForegroundStandard(ExistsBuilderImpl.java:240)
at
org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForeground(ExistsBuilderImpl.java:235)
at
org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:202)
at
org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:35)
at
org.apache.storm.zookeeper.ClientZookeeper.existsNode(ClientZookeeper.java:144)
... 7 more
o.a.s.e.EventManagerImp Thread-3 [ERROR] {} Error when processing event
java.lang.RuntimeException: java.lang.RuntimeException:
org.apache.storm.shade.org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for
/profilerconfigs/finanzen_net_forum-213-1790677956
at
org.apache.storm.daemon.supervisor.ReadClusterState.run(ReadClusterState.java:190)
at
org.apache.storm.daemon.supervisor.timer.SynchronizeAssignments.run(SynchronizeAssignments.java:86)
at org.apache.storm.event.EventManagerImp$1.run(EventManagerImp.java:49)
Caused by: java.lang.RuntimeException:
org.apache.storm.shade.org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for
/profilerconfigs/finanzen_net_forum-213-1790677956
at org.apache.storm.utils.Utils.wrapInRuntime(Utils.java:504)
at
org.apache.storm.zookeeper.ClientZookeeper.existsNode(ClientZookeeper.java:147)
at
org.apache.storm.cluster.ZKStateStorage.node_exists(ZKStateStorage.java:150)
at
org.apache.storm.cluster.PaceMakerStateStorage.node_exists(PaceMakerStateStorage.java:97)
at
org.apache.storm.cluster.StormClusterStateImpl.getTopologyProfileRequests(StormClusterStateImpl.java:366)
at
org.apache.storm.daemon.supervisor.ReadClusterState.getProfileActions(ReadClusterState.java:202)
at
org.apache.storm.daemon.supervisor.ReadClusterState.run(ReadClusterState.java:143)
... 2 more
Caused by:
org.apache.storm.shade.org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for
/profilerconfigs/finanzen_net_forum-213-1790677956
at
org.apache.storm.shade.org.apache.zookeeper.KeeperException.create(KeeperException.java:133)
at
org.apache.storm.shade.org.apache.zookeeper.KeeperException.create(KeeperException.java:53)
at
org.apache.storm.shade.org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1867)
at
org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl$3.call(ExistsBuilderImpl.java:247)
at
org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl$3.call(ExistsBuilderImpl.java:240)
at
org.apache.storm.shade.org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:88)
at
org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForegroundStandard(ExistsBuilderImpl.java:240)
at
org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForeground(ExistsBuilderImpl.java:235)
at
org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:202)
at
org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:35)
at
org.apache.storm.zookeeper.ClientZookeeper.existsNode(ClientZookeeper.java:144)
... 7 more
o.a.s.u.Utils Thread-3 [ERROR] Halting process: Error when processing an
event
java.lang.RuntimeException: Halting process: Error when processing an event
at org.apache.storm.utils.Utils.exitProcess(Utils.java:527)
at org.apache.storm.event.EventManagerImp$1.run(EventManagerImp.java:60)
o.a.s.u.Utils ShutdownHook-sleepKill-1s [WARN] Forcing Halt... \"process
reaper\" (DAEMON)
lock: java.util.concurrent.SynchronousQueue$Transferer@6f2e732 owner: null
java.lang.Thread.State: TIMED_WAITING
at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
at
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:408)
at
[email protected]/java.util.concurrent.LinkedTransferQueue$DualNode.await(LinkedTransferQueue.java:452)
at
[email protected]/java.util.concurrent.SynchronousQueue$Transferer.xferLifo(SynchronousQueue.java:194)
at
[email protected]/java.util.concurrent.SynchronousQueue.xfer(SynchronousQueue.java:235)
at
[email protected]/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:338)
at
[email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1015)
at
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1076)
at
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:614)
at [email protected]/java.lang.Thread.runWith(Thread.java:1487)
at [email protected]/java.lang.Thread.run(Thread.java:1474)
at
[email protected]/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:148)
\"pool-11-thread-144\"
lock:
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7b2f7977
owner: null
java.lang.Thread.State: WAITING
at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
at
[email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:369)
at
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(AbstractQueuedSynchronizer.java:520)
at
[email protected]/java.util.concurrent.ForkJoinPool.unmanagedBlock(ForkJoinPool.java:4364)
at
[email protected]/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:4310)
at
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1752)
at
[email protected]/java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:421)
at
[email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1016)
at
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1076)
at
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:614)
at [email protected]/java.lang.Thread.runWith(Thread.java:1487)
at [email protected]/java.lang.Thread.run(Thread.java:1474)
\"process reaper\" (DAEMON)
lock: java.util.concurrent.SynchronousQueue$Transferer@6f2e732 owner: null
java.lang.Thread.State: TIMED_WAITING
at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
at
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:408)
at
[email protected]/java.util.concurrent.LinkedTransferQueue$DualNode.await(LinkedTransferQueue.java:452)
at
[email protected]/java.util.concurrent.SynchronousQueue$Transferer.xferLifo(SynchronousQueue.java:194)
at
[email protected]/java.util.concurrent.SynchronousQueue.xfer(SynchronousQueue.java:235)
at
[email protected]/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:338)
at
[email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1015)
at
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1076)
at
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:614)
at [email protected]/java.lang.Thread.runWith(Thread.java:1487)
at [email protected]/java.lang.Thread.run(Thread.java:1474)
at
[email protected]/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:148)
\"pool-11-thread-178\"
lock:
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7b2f7977
owner: null
java.lang.Thread.State: WAITING
at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
at
[email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:369)
at
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(AbstractQueuedSynchronizer.java:520)
at
[email protected]/java.util.concurrent.ForkJoinPool.unmanagedBlock(ForkJoinPool.java:4364)
at
[email protected]/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:4310)
at
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1752)
at
[email protected]/java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:421)
at
[email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1016)
at
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1076)
at
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:614)
at [email protected]/java.lang.Thread.runWith(Thread.java:1487)
at [email protected]/java.lang.Thread.run(Thread.java:1474)
\"process reaper\" (DAEMON)
lock: java.util.concurrent.SynchronousQueue$Transferer@6f2e732 owner: null
java.lang.Thread.State: TIMED_WAITING
at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
at
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:408)
at
[email protected]/java.util.concurrent.LinkedTransferQueue$DualNode.await(LinkedTransferQueue.java:452)
at
[email protected]/java.util.concurrent.SynchronousQueue$Transferer.xferLifo(SynchronousQueue.java:194)
at
[email protected]/java.util.concurrent.SynchronousQueue.xfer(SynchronousQueue.java:235)
at
[email protected]/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:338)
at
[email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1015)
at
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1076)
at
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:614)
at [email protected]/java.lang.Thread.runWith(Thread.java:1487)
at [email protected]/java.lang.Thread.run(Thread.java:1474)
at
[email protected]/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:148)
\"process reaper\" (DAEMON)
lock: java.util.concurrent.SynchronousQueue$Transferer@6f2e732 owner: null
java.lang.Thread.State: TIMED_WAITING
at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
at
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:408)
at
[email protected]/java.util.concurrent.LinkedTransferQueue$DualNode.await(LinkedTransferQueue.java:452)
at
[email protected]/java.util.concurrent.SynchronousQueue$Transferer.xferLifo(SynchronousQueue.java:194)
at
[email protected]/java.util.concurrent.SynchronousQueue.xfer(SynchronousQueue.java:235)
at
[email protected]/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:338)
at
[email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1015)
at
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1076)
at
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:614)
at [email protected]/java.lang.Thread.runWith(Thread.java:1487)
at [email protected]/java.lang.Thread.run(Thread.java:1474)
at
[email protected]/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:148)
\"pool-11-thread-65\"
lock:
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7b2f7977
owner: null
java.lang.Thread.State: WAITING
at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
at
[email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:369)
at
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(AbstractQueuedSynchronizer.java:520)
at
[email protected]/java.util.concurrent.ForkJoinPool.unmanagedBlock(ForkJoinPool.java:4364)
at
[email protected]/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:4310)
at
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1752)
at
[email protected]/java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:421)
at
[email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1016)
at
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1076)
at
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:614)
at [email protected]/java.lang.Thread.runWith(Thread.java:1487)
at [email protected]/java.lang.Thread.run(Thread.java:1474)
\"SLOT_6916\"
lock: null owner: null
java.lang.Thread.State: TIMED_WAITING
at [email protected]/java.lang.Thread.sleepNanos0(Native Method)
at [email protected]/java.lang.Thread.sleepNanos(Thread.java:509)
at [email protected]/java.lang.Thread.sleep(Thread.java:540)
at app//org.apache.storm.utils.Time.sleep(Time.java:100)
at
app//org.apache.storm.daemon.supervisor.Slot.handleRunning(Slot.java:882)
at
app//org.apache.storm.daemon.supervisor.Slot.stateMachineStep(Slot.java:184)
at app//org.apache.storm.daemon.supervisor.Slot.run(Slot.java:1051)
\"process reaper\" (DAEMON)
lock: java.util.concurrent.SynchronousQueue$Transferer@6f2e732 owner: null
java.lang.Thread.State: TIMED_WAITING
at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
at
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:408)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
