[
https://issues.apache.org/jira/browse/STORM-822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275980#comment-15275980
]
ASF GitHub Bot commented on STORM-822:
--------------------------------------
Github user jianbzhou commented on the pull request:
https://github.com/apache/storm/pull/1131#issuecomment-217781379
Just think another scenario – for example we polled 10k message and emitted
– say from 10001 to 20000, so the numUncommittedOffsets is 10000, if the 10500
msg failed, which caused all the following message 10501~20000 will not got
commited until the 10500 message was reemitted and acked.
If a rebalance happened during this time, the offset will seek back to the
last commited offiset +1, possibly will seek back to offset 10001, then all the
message from 10001 to 2000 will be polled and emitted again – which will cause
numUncommittedOffsets be incremented by another 10000 again.
After successfully commit to kafka, numUncommittedOffsets will substract
10000 and leave 10000 value there.
Seems this will also gradually cause numUncommittedOffsets be a bigger
value than we expect, Is this a possible scenario?
> Kafka Spout New Consumer API
> ----------------------------
>
> Key: STORM-822
> URL: https://issues.apache.org/jira/browse/STORM-822
> Project: Apache Storm
> Issue Type: Story
> Components: storm-kafka
> Reporter: Thomas Becker
> Assignee: Hugo Louro
> Fix For: 1.0.0, 2.0.0
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
