[ 
https://issues.apache.org/jira/browse/STORM-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stig Rohde Døssing updated STORM-2691:
--------------------------------------
    Description: 
The Trident Kafka spout uses the KafkaTridentSpoutTopicPartitionRegistry enum 
to pass existing topic partitions from the spout to the coordinator. This only 
works when those components happen to be in the same JVM, because the 
coordinator gets the topic information from the KafkaConsumer started by 
KafkaTridentSpoutEmitter. 

The coordinator runs in the TridentSpoutCoordinator bolt here 
https://github.com/apache/storm/blob/4c8a986f519cdf3e63bed47e9c4f723e4867267a/storm-client/src/jvm/org/apache/storm/trident/topology/TridentTopologyBuilder.java#L162,
 while the spout instances (emitters) run in TridentSpoutExecutors here 
https://github.com/apache/storm/blob/4c8a986f519cdf3e63bed47e9c4f723e4867267a/storm-client/src/jvm/org/apache/storm/trident/topology/TridentTopologyBuilder.java#L176.

We should replace the registry enum with writes to Zookeeper or something 
similar.

  was:
The Trident Kafka spout uses the KafkaTridentSpoutTopicPartitionRegistry enum 
to pass existing topic partitions from the spout to the coordinator. This only 
works when those components happen to be in the same JVM. 

The coordinator runs in the TridentSpoutCoordinator bolt here 
https://github.com/apache/storm/blob/4c8a986f519cdf3e63bed47e9c4f723e4867267a/storm-client/src/jvm/org/apache/storm/trident/topology/TridentTopologyBuilder.java#L162,
 while the spout instances run in TridentSpoutExecutors here 
https://github.com/apache/storm/blob/4c8a986f519cdf3e63bed47e9c4f723e4867267a/storm-client/src/jvm/org/apache/storm/trident/topology/TridentTopologyBuilder.java#L176.

We should replace the registry enum with writes to Zookeeper or something 
similar.


> storm-kafka-client Trident spout communicates with the Coordinator through a 
> static field
> -----------------------------------------------------------------------------------------
>
>                 Key: STORM-2691
>                 URL: https://issues.apache.org/jira/browse/STORM-2691
>             Project: Apache Storm
>          Issue Type: Bug
>          Components: storm-kafka-client
>    Affects Versions: 2.0.0, 1.1.1
>            Reporter: Stig Rohde Døssing
>
> The Trident Kafka spout uses the KafkaTridentSpoutTopicPartitionRegistry enum 
> to pass existing topic partitions from the spout to the coordinator. This 
> only works when those components happen to be in the same JVM, because the 
> coordinator gets the topic information from the KafkaConsumer started by 
> KafkaTridentSpoutEmitter. 
> The coordinator runs in the TridentSpoutCoordinator bolt here 
> https://github.com/apache/storm/blob/4c8a986f519cdf3e63bed47e9c4f723e4867267a/storm-client/src/jvm/org/apache/storm/trident/topology/TridentTopologyBuilder.java#L162,
>  while the spout instances (emitters) run in TridentSpoutExecutors here 
> https://github.com/apache/storm/blob/4c8a986f519cdf3e63bed47e9c4f723e4867267a/storm-client/src/jvm/org/apache/storm/trident/topology/TridentTopologyBuilder.java#L176.
> We should replace the registry enum with writes to Zookeeper or something 
> similar.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Reply via email to