GitHub user srdo opened a pull request:
https://github.com/apache/storm/pull/1340
STORM-1682: Refactor DynamicBrokersReader to lookup leader informatioâ¦
â¦n via Kafka metadata API. Put Zookeeper metadata lookup into new class.
See https://issues.apache.org/jira/browse/STORM-1682.
This solution splits DynamicBrokersReader into two. Instead of looking up
leaders and brokers in Zookeeper, ZkBrokersReader will use the brokers in
Zookeeper to construct one or more SimpleConsumers, which DynamicBrokersReader
can then use to lookup partition leaders.
DynamicBrokersReader should now cause the spout to restart in the following
scenarios:
If it can't find any brokers in Zookeeper
If it can't contact any of the brokers it found in Zookeeper
If wildcard topics are listed in Zookeeper, and no broker has metadata for
all of them
In order to test behavior when brokers shut down, it was necessary to
separate the Kafka test broker from Zookeeper so multiple brokers can run in a
cluster, which is the reason for modifications in many tests.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/srdo/storm STORM-1682
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/1340.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1340
----
commit 78a3d0683d1bfbfec7bbc7425927347b4b47154d
Author: Stig Rohde Døssing <[email protected]>
Date: 2016-04-12T18:24:49Z
STORM-1682: Refactor DynamicBrokersReader to lookup leader information via
Kafka metadata API. Put Zookeeper metadata lookup into new class.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
