GitHub user spelliott opened a pull request:
https://github.com/apache/incubator-storm/pull/170
Kafka spout: throw RuntimeException if a leader cannot be found for a
partition
If one of the partitions of the Kafka queue has no leader available,
DynamicBrokersReader.getLeaderFor(partition) returns -1. This is logged in
DynamicBrokersReader.getBrokerInfo (as there's no ZK node for a broker id -1)
but there is no further action.
In this situation the spout will carry on emitting only for those
partitions that are available.
My assumption is that if some partitions for a queue are unavailable,
there's a problem that needs to be addressed before processing continues.
Happy to discuss further.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/spelliott/incubator-storm kafka-no-leader
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-storm/pull/170.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #170
----
commit 974f0f377a371db6a673c59e13a9640fbdd9d92a
Author: Stephen Elliott <[email protected]>
Date: 2014-06-30T10:25:49Z
Throw RuntimeException if a leader cannot be found for a partition
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
