[
https://issues.apache.org/jira/browse/STORM-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guang Du updated STORM-2666:
----------------------------
Description:
Under a certain heavy load, for failed/timeout tuples, the retry service will
ack tuple for failed max times. Kafka Client Spout will commit after reached
the commit interval. However seems some 'on the way' tuples will be failed
again, the retry service will cause Spout to emit again, and acked eventually
to OffsetManager.
In some cases such offsets are too many, exceeding the max-uncommit, causing
org.apache.storm.kafka.spout.internal.OffsetManager#findNextCommitOffset unable
to find next commit point, and Spout for this partition will not poll any more.
resulting logs like below. I'm not sure if the issue has already been
raised/fixed, glad if anyone could help to point out existing JIRA. Thank you.
2017-07-27 22:23:48.398 o.a.s.k.s.KafkaSpout Thread-23-spout-executor[248 248]
[INFO] Successful ack for tuple message
[{topic-partition=kafka_bd_trigger_action-20, offset=18204, numFails=0}].
2017-07-27 22:23:49.203 o.a.s.k.s.i.OffsetManager Thread-23-spout-executor[248
248] [WARN] topic-partition [kafka_bd_trigger_action-18] has unexpected offset
[16002]. Current committed Offset [16003]
was:
Under a certain heavy load, for failed/timeout tuples, the retry service will
ack tuple for failed max times. Kafka Client Spout will commit after reached
the commit interval. However seems some 'on the way' tuples will be resent
again by Spout, and acked eventually to OffsetManager, resulting logs like
below. I'm not sure if the issue has already been raised/fixed, glad if anyone
could help to point out existing JIRA. Thank you.
2017-07-27 22:23:48.398 o.a.s.k.s.KafkaSpout Thread-23-spout-executor[248 248]
[INFO] Successful ack for tuple message
[{topic-partition=kafka_bd_trigger_action-20, offset=18204, numFails=0}].
2017-07-27 22:23:49.203 o.a.s.k.s.i.OffsetManager Thread-23-spout-executor[248
248] [WARN] topic-partition [kafka_bd_trigger_action-18] has unexpected offset
[16002]. Current committed Offset [16003]
> Kafka Client Spout send & ack committed offsets
> -----------------------------------------------
>
> Key: STORM-2666
> URL: https://issues.apache.org/jira/browse/STORM-2666
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-kafka-client
> Affects Versions: 1.1.1
> Reporter: Guang Du
>
> Under a certain heavy load, for failed/timeout tuples, the retry service will
> ack tuple for failed max times. Kafka Client Spout will commit after reached
> the commit interval. However seems some 'on the way' tuples will be failed
> again, the retry service will cause Spout to emit again, and acked eventually
> to OffsetManager.
> In some cases such offsets are too many, exceeding the max-uncommit, causing
> org.apache.storm.kafka.spout.internal.OffsetManager#findNextCommitOffset
> unable to find next commit point, and Spout for this partition will not poll
> any more.
> resulting logs like below. I'm not sure if the issue has already been
> raised/fixed, glad if anyone could help to point out existing JIRA. Thank you.
> 2017-07-27 22:23:48.398 o.a.s.k.s.KafkaSpout Thread-23-spout-executor[248
> 248] [INFO] Successful ack for tuple message
> [{topic-partition=kafka_bd_trigger_action-20, offset=18204, numFails=0}].
> 2017-07-27 22:23:49.203 o.a.s.k.s.i.OffsetManager
> Thread-23-spout-executor[248 248] [WARN] topic-partition
> [kafka_bd_trigger_action-18] has unexpected offset [16002]. Current committed
> Offset [16003]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
