[
https://issues.apache.org/jira/browse/KAFKA-9543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17041716#comment-17041716
]
Brian Jones commented on KAFKA-9543:
------------------------------------
I think we've hit the same issue after upgrading from Kafka 2.2.1 to Kafka
2.4.0..
The "Fetch offset nnn is out of range for partition xxx, resetting offset" only
ever appears in our logs after the upgrade. We've actually downgraded back to
2.2.1 and have seen no more re-occurrences of the issue.
>From what we've observed:
* This does NOT happen on every segment rollover.
* All of the topics we've seen it occur on are compacted topics.
* The start offset for the topic partitions in question have often been 0, so
the only place the offset can be out of range is at the end.
* We saw it happen with two different topics, and more frequently on the topic
with higher throughput.
* It only ever occurred in our production environment, where we would have
higher throughput.
It is a Kafka Streams application consuming from topic, which resulted in it
re-producing a load of downstream messages so we had to revert to the older
version. We don't use the "exactly once" processing guarantee in this
application, but presumably this would've completely broken that guarantee, too.
> Consumer offset reset after new segment rolling
> -----------------------------------------------
>
> Key: KAFKA-9543
> URL: https://issues.apache.org/jira/browse/KAFKA-9543
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 2.4.0
> Reporter: Rafał Boniecki
> Priority: Major
> Attachments: Untitled.png
>
>
> After upgrade from kafka 2.1.1 to 2.4.0, I'm experiencing unexpected consumer
> offset resets.
> Consumer:
> {code:java}
> 2020-02-12T11:12:58.402+01:00 hostname 4a2a39a35a02
> [2020-02-12T11:12:58,402][INFO
> ][org.apache.kafka.clients.consumer.internals.Fetcher] [Consumer
> clientId=logstash-1, groupId=logstash] Fetch offset 1632750575 is out of
> range for partition stats-5, resetting offset
> {code}
> Broker:
> {code:java}
> 2020-02-12 11:12:58:400 CET INFO
> [data-plane-kafka-request-handler-1][kafka.log.Log] [Log partition=stats-5,
> dir=/kafka4/data] Rolled new log segment at offset 1632750565 in 2 ms.{code}
> All resets are perfectly correlated to rolling new segments at the broker -
> segment is rolled first, then, couple of ms later, reset on the consumer
> occurs. Attached is grafana graph with consumer lag per partition. All sudden
> spikes in lag are offset resets due to this bug.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
