Haruki Okada created KAFKA-12261:
------------------------------------
Summary: auto.offset.reset should be earliest by default
Key: KAFKA-12261
URL: https://issues.apache.org/jira/browse/KAFKA-12261
Project: Kafka
Issue Type: Improvement
Affects Versions: 2.7.0
Reporter: Haruki Okada
As of now, auto.offset.reset of ConsumerConfig is "latest" by default.
This could be a pitfall that causes message delivery loss when we split topic's
partitions like below:
Say we have a topic-X which have only 1 partition.
# split topic-X to 2 partitions by kafka-topics.sh --alter --topic topic-X
--partitions 2 (topic-X-1 is added)
# producer knows that new partitions are added by refreshing metadata. starts
to produce to topic-X-1
# bit later, consumer knows that new partitions are added and triggering
consumer rebalance, then starts consuming topic-X-1
** upon starting consumption, it resets its offset to log-end-offset
If the producer sent several records before 3, they could be not-delivered to
the consumer.
This behavior isn't preferable in most cases, so auto.offset.reset should be
set to "earliest" by default to avoid this pitfall.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
