Colin Shaw created KAFKA-14739: ---------------------------------- Summary: Kafka consumer reading messages out of order after a rebalance Key: KAFKA-14739 URL: https://issues.apache.org/jira/browse/KAFKA-14739 Project: Kafka Issue Type: Bug Affects Versions: 2.6.0 Reporter: Colin Shaw Attachments: image-2023-02-21-16-45-10-464.png
We are seeing often when our consumers are running and a rebalance is triggered, some partitions do not resume at the correct offset causing a large number of events to be skipped. Our understanding is that within a partition events should also be read in chronological order. These consumers run once a day and we see that when they resume again the next day it does return to the missed events. An example we investigated: {quote}We see for partition 88 starting processing around Offset 14001. Events were processed until Offset 14059 but we could see Offset 14060 was not processed from out logs. Upon investigation we could see a rebalance had occured after offset 14059 and afterwards the next event processed was offset 14190 causing about 130 events to be skipped on this partition. The subsequent day we it did resume at Offset 14060 {quote} -- This message was sent by Atlassian Jira (v8.20.10#820010)