[
https://issues.apache.org/jira/browse/STORM-3047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stig Rohde Døssing updated STORM-3047:
--------------------------------------
Description:
This is a backport of the changes made to OpaquePartitionedTridentSpoutExecutor
in https://github.com/apache/storm/pull/2300/files.
The description of the issue is copied here for convenience:
The changes in https://github.com/apache/storm/pull/2009 released in 1.1.0 made
some changes to the OpaquePartitionedTridentSpoutExecutor that likely broke
IOpaquePartitionedTridentSpout implementations other than storm-kafka-client.
The changed code used to request sorted partitions from the spout via
getOrderedPartitions, do a round-robin partitioning, and assign partitions via
refreshPartitions
https://github.com/apache/storm/blob/v1.0.4/storm-core/src/jvm/org/apache/storm/trident/spout/OpaquePartitionedTridentSpoutExecutor.java#L100.
The new code just passes the output of getOrderedPartitions into
refreshPartitions
https://github.com/apache/storm/blob/v1.1.0/storm-core/src/jvm/org/apache/storm/trident/spout/OpaquePartitionedTridentSpoutExecutor.java#L120.
It looks to me like refreshPartitions is passed the list of all partitions
assigned to any spout task, rather than just the partitions assigned to the
current task.
The proposed fix will use getOrderedPartitions to get the sorted partitions
list, pass the list into getPartitionsForTask, and pass the resulting list of
assigned partitions back into refreshPartitions.
was:This is a backport of the changes made to
OpaquePartitionedTridentSpoutExecutor in
https://github.com/apache/storm/pull/2300/files.
> Ensure Trident emitter refreshPartitions is only called with partitions
> assigned to the emitter
> -----------------------------------------------------------------------------------------------
>
> Key: STORM-3047
> URL: https://issues.apache.org/jira/browse/STORM-3047
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-core
> Affects Versions: 1.1.2, 1.2.1
> Reporter: Stig Rohde Døssing
> Assignee: Stig Rohde Døssing
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This is a backport of the changes made to
> OpaquePartitionedTridentSpoutExecutor in
> https://github.com/apache/storm/pull/2300/files.
> The description of the issue is copied here for convenience:
> The changes in https://github.com/apache/storm/pull/2009 released in 1.1.0
> made some changes to the OpaquePartitionedTridentSpoutExecutor that likely
> broke IOpaquePartitionedTridentSpout implementations other than
> storm-kafka-client. The changed code used to request sorted partitions from
> the spout via getOrderedPartitions, do a round-robin partitioning, and assign
> partitions via refreshPartitions
> https://github.com/apache/storm/blob/v1.0.4/storm-core/src/jvm/org/apache/storm/trident/spout/OpaquePartitionedTridentSpoutExecutor.java#L100.
> The new code just passes the output of getOrderedPartitions into
> refreshPartitions
> https://github.com/apache/storm/blob/v1.1.0/storm-core/src/jvm/org/apache/storm/trident/spout/OpaquePartitionedTridentSpoutExecutor.java#L120.
> It looks to me like refreshPartitions is passed the list of all partitions
> assigned to any spout task, rather than just the partitions assigned to the
> current task.
> The proposed fix will use getOrderedPartitions to get the sorted partitions
> list, pass the list into getPartitionsForTask, and pass the resulting list of
> assigned partitions back into refreshPartitions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)