Pedro Azevedo created STORM-4077:
------------------------------------
Summary: Worker being reassigned when Nimbus leadership changes
Key: STORM-4077
URL: https://issues.apache.org/jira/browse/STORM-4077
Project: Apache Storm
Issue Type: Bug
Affects Versions: 2.6.1
Reporter: Pedro Azevedo
Hey guys, I'm using Storm v2.6.1 and every time I restart the nimbus leader
(currently I have 3 for high availability) the workers get reassigned and this
is a bad behaviour as every topology will have no workers running for a certain
period(until new workers are assigned) due to a Nimbus leadership change.
On another note, when stopping the nimbus I'm getting this error which seems to
be impacting the gracefully shutdown.
2024-08-21T15:09:46.647Z Nimbus [INFO] Shutting down master
2024-08-21T15:09:46.648Z CuratorFrameworkImpl [INFO] backgroundOperationsLoop
exiting 2024-08-21T15:09:46.752Z ClientCnxn [INFO] EventThread shut down for
session: 0x4000010be7caa5d 2024-08-21T15:09:46.752Z ZooKeeper [INFO] Session:
0x4000010be7caa5d closed 2024-08-21T15:09:46.752Z CuratorFrameworkImpl [INFO]
backgroundOperationsLoop exiting 2024-08-21T15:09:46.812Z ProcessFunction
[ERROR] Internal error processing getLeader java.lang.IllegalStateException:
Expected state [STARTED] was [STOPPED] at
org.apache.storm.shade.org.apache.curator.shaded.com.google.common.base.Preconditions.checkState(Preconditions.java:835)
~[storm-shaded-deps-2.6.1.jar:2.6.1] at
org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.checkState(CuratorFrameworkImpl.java:462)
~[storm-shaded-deps-2.6.1.jar:2.6.1] at
org.apache.storm.shade.org.apache.curator.framework.imps.CuratorFrameworkImpl.getChildren(CuratorFrameworkImpl.java:507)
~[storm-shaded-deps-2.6.1.jar:2.6.1] at
org.apache.storm.zookeeper.ClientZookeeper.getChildren(ClientZookeeper.java:209)
~[storm-client-2.6.1.jar:2.6.1] at
org.apache.storm.cluster.ZKStateStorage.get_children(ZKStateStorage.java:155)
~[storm-client-2.6.1.jar:2.6.1] at
org.apache.storm.cluster.StormClusterStateImpl.nimbuses(StormClusterStateImpl.java:279)
~[storm-client-2.6.1.jar:2.6.1] at
org.apache.storm.daemon.nimbus.Nimbus.getLeader(Nimbus.java:4907)
~[storm-server-2.6.1.jar:2.6.1] at
org.apache.storm.generated.Nimbus$Processor$getLeader.getResult(Nimbus.java:5168)
~[storm-client-2.6.1.jar:2.6.1] at
org.apache.storm.generated.Nimbus$Processor$getLeader.getResult(Nimbus.java:5144)
~[storm-client-2.6.1.jar:2.6.1] at
org.apache.storm.thrift.ProcessFunction.process(ProcessFunction.java:40)
[storm-shaded-deps-2.6.1.jar:2.6.1] at
org.apache.storm.thrift.TBaseProcessor.process(TBaseProcessor.java:40)
[storm-shaded-deps-2.6.1.jar:2.6.1] at
org.apache.storm.security.auth.SimpleTransportPlugin$SimpleWrapProcessor.process(SimpleTransportPlugin.java:171)
[storm-client-2.6.1.jar:2.6.1] at
org.apache.storm.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:492)
[storm-shaded-deps-2.6.1.jar:2.6.1] at
org.apache.storm.thrift.server.Invocation.run(Invocation.java:19)
[storm-shaded-deps-2.6.1.jar:2.6.1] at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[?:?] at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[?:?] at java.base/java.lang.Thread.run(Thread.java:829) [?:?]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
