[
https://issues.apache.org/jira/browse/STORM-380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14960955#comment-14960955
]
Francois Visconte commented on STORM-380:
-----------------------------------------
Hi,
We have a SLA that makes offline partition possible from time to time. This
cause our topology to fail repeateadly in this case preventing processing of
all other partitions.
What about adding a configuration parameter to allow a mode where offline
partition are logged as warning and re-integrated in the next
ZkCoordinator.refresh().
Here is the commit I made to fix our problem but I can add a config param for
that.
Cheers,
F.
> Kafka spout: throw RuntimeException if a leader cannot be found for a
> partition
> -------------------------------------------------------------------------------
>
> Key: STORM-380
> URL: https://issues.apache.org/jira/browse/STORM-380
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-kafka
> Reporter: Stephen Elliott
> Assignee: Stephen Elliott
> Priority: Minor
> Fix For: 0.9.3
>
>
> If one of the partitions of the Kafka queue has no leader available,
> DynamicBrokersReader.getLeaderFor(partition) returns -1. This is logged in
> DynamicBrokersReader.getBrokerInfo (as there's no ZK node for a broker id -1)
> but there is no further action.
> In this situation the spout will carry on emitting only for those partitions
> that are available.
> My assumption is that if some partitions for a queue are unavailable, there's
> a problem that needs to be addressed before processing continues. Happy to
> discuss further.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
