[
https://issues.apache.org/jira/browse/STORM-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jungtaek Lim resolved STORM-2296.
---------------------------------
Resolution: Fixed
Assignee: Ernestas Vaiciukevičius
Fix Version/s: 1.1.0
2.0.0
Thanks [~ernisv], I merged into master and 1.x branch.
I don't merge this to 1.0.x branch since this is rather improvement, not a
bugfix. If you feel this would be better to be applied to 1.0.x branch, please
let me know.
> Kafka spout - no duplicates on leader changes
> ---------------------------------------------
>
> Key: STORM-2296
> URL: https://issues.apache.org/jira/browse/STORM-2296
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-kafka
> Affects Versions: 1.0.2
> Reporter: Ernestas Vaiciukevičius
> Assignee: Ernestas Vaiciukevičius
> Fix For: 2.0.0, 1.1.0
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> Current behavior of Kafka spout emits duplicate tuples whenever Kafka topic
> leader's change.
> In case of exception caused by leader changes, PartitionManagers are simply
> recreated losing the state about which tuples were already emitted and new
> PartitionManager re-emits them again.
> This is fine as at-least-once is fulfilled, but still it would be better to
> not emit duplicate data if possible.
> Moreover this could be easily avoided by moving the state related to emitted
> tuples from old PartitionManager to new one.
> Pull requests implementing this:
> 1.0.x-branch - https://github.com/apache/storm/pull/1873
> 1.x-branch - https://github.com/apache/storm/pull/1888
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
