[ 
https://issues.apache.org/jira/browse/STORM-3047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stig Rohde Døssing updated STORM-3047:
--------------------------------------
    Description: 
This is a backport of the changes made to OpaquePartitionedTridentSpoutExecutor 
in https://github.com/apache/storm/pull/2300/files.

The description of the issue is copied here for convenience:

The changes in https://github.com/apache/storm/pull/2009 released in 1.1.0 made 
some changes to the OpaquePartitionedTridentSpoutExecutor that likely broke 
IOpaquePartitionedTridentSpout implementations other than storm-kafka-client. 
The changed code used to request sorted partitions from the spout via 
getOrderedPartitions, do a round-robin partitioning, and assign partitions via 
refreshPartitions 
https://github.com/apache/storm/blob/v1.0.4/storm-core/src/jvm/org/apache/storm/trident/spout/OpaquePartitionedTridentSpoutExecutor.java#L100.
 The new code just passes the output of getOrderedPartitions into 
refreshPartitions 
https://github.com/apache/storm/blob/v1.1.0/storm-core/src/jvm/org/apache/storm/trident/spout/OpaquePartitionedTridentSpoutExecutor.java#L120.
 It looks to me like refreshPartitions is passed the list of all partitions 
assigned to any spout task, rather than just the partitions assigned to the 
current task.

The proposed fix will use getOrderedPartitions to get the sorted partitions 
list, pass the list into getPartitionsForTask, and pass the resulting list of 
assigned partitions back into refreshPartitions.


  was:This is a backport of the changes made to 
OpaquePartitionedTridentSpoutExecutor in 
https://github.com/apache/storm/pull/2300/files.


> Ensure Trident emitter refreshPartitions is only called with partitions 
> assigned to the emitter
> -----------------------------------------------------------------------------------------------
>
>                 Key: STORM-3047
>                 URL: https://issues.apache.org/jira/browse/STORM-3047
>             Project: Apache Storm
>          Issue Type: Bug
>          Components: storm-core
>    Affects Versions: 1.1.2, 1.2.1
>            Reporter: Stig Rohde Døssing
>            Assignee: Stig Rohde Døssing
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> This is a backport of the changes made to 
> OpaquePartitionedTridentSpoutExecutor in 
> https://github.com/apache/storm/pull/2300/files.
> The description of the issue is copied here for convenience:
> The changes in https://github.com/apache/storm/pull/2009 released in 1.1.0 
> made some changes to the OpaquePartitionedTridentSpoutExecutor that likely 
> broke IOpaquePartitionedTridentSpout implementations other than 
> storm-kafka-client. The changed code used to request sorted partitions from 
> the spout via getOrderedPartitions, do a round-robin partitioning, and assign 
> partitions via refreshPartitions 
> https://github.com/apache/storm/blob/v1.0.4/storm-core/src/jvm/org/apache/storm/trident/spout/OpaquePartitionedTridentSpoutExecutor.java#L100.
>  The new code just passes the output of getOrderedPartitions into 
> refreshPartitions 
> https://github.com/apache/storm/blob/v1.1.0/storm-core/src/jvm/org/apache/storm/trident/spout/OpaquePartitionedTridentSpoutExecutor.java#L120.
>  It looks to me like refreshPartitions is passed the list of all partitions 
> assigned to any spout task, rather than just the partitions assigned to the 
> current task.
> The proposed fix will use getOrderedPartitions to get the sorted partitions 
> list, pass the list into getPartitionsForTask, and pass the resulting list of 
> assigned partitions back into refreshPartitions.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to