Workers communicate to nimbus though ZooKeeper. If ZK is under a lot of load
and is not able to handle all of the write requests, nimbus can think that the
workers are not heart-beating, even when they are. If this happens it will
reschedule them, possibly on a different node/location. If that happens the
Supervisors will dutifully obey and change things around to match the new
scheduling. If you rebalance your topology, nimbus lies to the scheduler
telling it that none of the workers for your topology are running, and lets it
pick all new places for them to run. Then the supervisors shoot/launch workers
to make things match the new scheduling.
- Bobby
On Friday, November 14, 2014 12:11 PM, Yashwant Ganti
<[email protected]> wrote:
Hello All,
We had a problem with Zookeeper infrastructure recently and the components in
our topology would face timeouts when trying to establish a connection. After a
prolonged period of attempting to connect and throwing exceptions, I noticed
that some of the workers were restarted. One of the other devs mentioned that
this was because the topology had to be 'rebalanced'. I did not quite
understand this and was wondering if someone could shed light on why storm
decided to restart the workers.
Thanks,Yash
