Han You created FLINK-41270:
-------------------------------
Summary: [Kafka] Explicit auto.offset.reset silently changes
committedOffsets() startup position
Key: FLINK-41270
URL: https://issues.apache.org/jira/browse/FLINK-41270
Project: Flink
Issue Type: Bug
Components: Connectors / Kafka
Affects Versions: kafka-5.1.0
Reporter: Han You
Since FLINK-39888, an explicitly configured auto.offset.reset is used as
is. The builder warning and docs say the starting offsets initializer still
selects the initial position, and auto.offset.reset only applies if that
position later becomes unavailable.
That does not hold for OffsetsInitializer.committedOffsets(strategy). Its
splits start at COMMITTED_OFFSET, and the reader leaves the position to the
KafkaConsumer. For a partition with no committed offset, the consumer
applies auto.offset.reset, so the explicit property, not the initializer's
strategy, decides where the partition starts.
Example: committedOffsets(OffsetResetStrategy.EARLIEST) with
auto.offset.reset=latest. A new topic matched by the pattern starts at
latest, and records written before the reader was assigned are skipped.
The builder logs "The source will use the initializer for startup", which
is not what happens.
Suggested fix: for ReaderHandledOffsetsInitializer (committedOffsets),
reword the warning and the docs to say that auto.offset.reset also decides
where partitions without a committed offset start.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
