Github user clockfly commented on the pull request:
https://github.com/apache/storm/pull/268#issuecomment-59674932
To retain the comments, I modified ted's branch.
The update will tries to slove these three problems:
1. When target worker is down, the source worker should be aware of this
and should not crash.
The approach we use here is as follows:
a. when target worker is down
b. the source worker will begin reconnect (It can retry upto
"storm.messaging.netty.max_retries" times)
c. source worker know that target worker is down(by watching zookeeper
nodes)
d. source worker call target client.close(), client.close() now will set
the flag closing as true asyncly, and this will break the reconnection at step
b.
e. reconnection aborted. No exception thrown.
2. When target worker is down, the data sending to other target worker
should not be blocked.
The approach we currently using is to drop messages when connection to
target worker is not available.
3. There is a side effect of solution in 2. During the course of topology
setup, will we drop the messages if some target worker connection is
establishing?
For this problem, we will now wait the connection to be ready, before
bringing up spout/bolts during worker startup. So, during the worker start, the
spout/bolts are activated till the connections to all target worker is
established.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
