[
https://issues.apache.org/jira/browse/STORM-2719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16152046#comment-16152046
]
Janith Kaiprath Valiyalappil commented on STORM-2719:
-----------------------------------------------------
Hi [~Srdo], I believe its the same issue. My bad, should have checked for
duplicates.
> Trident Kafka Spout Emitters do not get full partition information in
> getOrderedPartitions()
> --------------------------------------------------------------------------------------------
>
> Key: STORM-2719
> URL: https://issues.apache.org/jira/browse/STORM-2719
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-kafka-client, trident
> Affects Versions: 1.1.1
> Reporter: Janith Kaiprath Valiyalappil
>
> The storm kakfa trident spout uses the
> KafkaTridentSpoutTopicPartitionRegistry, to get partition information. The
> coordinator calls the getTopicPartitions() method to get partition
> information and passes it to the emitters. But this partition information
> will not be accurate as all instances of
> KafkaTridentSpoutTopicPartitionRegistry will not be updated with full
> partition information.
> The update to the registry is done when the consumer subscribes using
> KafkaSpoutConsumerRebalanceListener. This calls the
> KafkaTridentSpoutTopicPartitionRegistry.INSTANCE.addAll(partitions); These
> calls would only update the registry in that particular worker with partition
> information for consumers in that worker.
> So when the coordinator calls the getOrderedPartitions() and passes it to
> each emitter by calling getOrderedPartitions(), the full partition
> information will not be present. The only probable case this would work is if
> the emitters and coordinators were on the same worker.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
