[
https://issues.apache.org/jira/browse/STORM-2608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jungtaek Lim resolved STORM-2608.
---------------------------------
Resolution: Fixed
Assignee: Kevin Conaway
Fix Version/s: 1.2.0
1.1.1
2.0.0
Thanks [~kevinconaway], I merged into master, 1.x, 1.1.x branches.
> Out Of Range Offsets Should Be Removed From Pending Queue
> ---------------------------------------------------------
>
> Key: STORM-2608
> URL: https://issues.apache.org/jira/browse/STORM-2608
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-kafka
> Affects Versions: 0.10.0, 1.0.4, 1.1.1
> Reporter: Kevin Conaway
> Assignee: Kevin Conaway
> Fix For: 2.0.0, 1.1.1, 1.2.0
>
> Time Spent: 4h
> Remaining Estimate: 0h
>
> There is a bug that occurs when failed tuples are invalidated due to Kafka
> throwing a {{TopicOffsetOutOfRangeException}}.
> Below is what happens:
> - Spout emits tuples
> - Offsets are added to the _pending_ tree
> - Some tuples fail and are added to the _failedMsgRetryManager_
> - On the next fetch request, a {{TopicOffsetOutOfRangeException}} is thrown
> and the new offset is _after_ the offset that are currently sitting in both
> the _pending_ tree and the _failedMsgRetryManager_
> - All offsets smaller than the the new offset are removed from the
> _failedMsgRetryManager_ but *not* the _pending_ tree.
> - Since those offsets were removed from the _failedMsgRetryManager_ they will
> never be retried and thus never get removed from _pending_
> - {{lastCommittedOffset()}} will always return the same value which means
> that offset in zookeeper for that partition will never get updated.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
