[
https://issues.apache.org/jira/browse/STORM-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14249521#comment-14249521
]
ASF GitHub Bot commented on STORM-391:
--------------------------------------
Github user harshach commented on the pull request:
https://github.com/apache/storm/pull/338#issuecomment-67280407
@Lewuathe can you give details whats the agreed upon implementation here.
We could use curator to use kafka zk /topics to check if the given topic
exists or not. The problem here is we are assuming that user always going to
give kafka zookeeper hosts which is not always true. User can just give kafka
broker hosts and storm zookeeper hosts for storing offset info. In this case
checking zk path for /topics won't work. Thats my concern about using zk path
check.
> KafkaSpout to await for the topic
> ---------------------------------
>
> Key: STORM-391
> URL: https://issues.apache.org/jira/browse/STORM-391
> Project: Apache Storm
> Issue Type: Improvement
> Affects Versions: 0.9.2-incubating
> Reporter: Alexey Raga
> Assignee: Kai Sasaki
> Labels: features
>
> When topic does not yet exist and the consumer is asked to consume from it,
> the default behaviour for Kafka heigh-level consumer is to "await" for the
> topic without a failure.
> KafkaSpout currently fails trying to get the partition information about the
> topic that does not exist.
> It may be a good idea to have the same common behaviour in KafkaSpout and it
> can probably be implemented through the zookeeper watchers: if topic does not
> exist, then set up a watcher and don't do anything until it yields.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
