[
https://issues.apache.org/jira/browse/FLINK-41270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-41270:
-----------------------------------
Labels: pull-request-available (was: )
> [Kafka] Explicit auto.offset.reset silently changes committedOffsets()
> startup position
> ---------------------------------------------------------------------------------------
>
> Key: FLINK-41270
> URL: https://issues.apache.org/jira/browse/FLINK-41270
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
> Affects Versions: kafka-5.1.0
> Reporter: Han You
> Priority: Minor
> Labels: pull-request-available
>
> Since FLINK-39888, an explicitly configured auto.offset.reset is used as
> is. The builder warning and docs say the starting offsets initializer still
> selects the initial position, and auto.offset.reset only applies if that
> position later becomes unavailable.
> That does not hold for OffsetsInitializer.committedOffsets(strategy). Its
> splits start at COMMITTED_OFFSET, and the reader leaves the position to the
> KafkaConsumer. For a partition with no committed offset, the consumer
> applies auto.offset.reset, so the explicit property, not the initializer's
> strategy, decides where the partition starts.
> Example: committedOffsets(OffsetResetStrategy.EARLIEST) with
> auto.offset.reset=latest. A new topic matched by the pattern starts at
> latest, and records written before the reader was assigned are skipped.
> The builder logs "The source will use the initializer for startup", which
> is not what happens.
> Suggested fix: for ReaderHandledOffsetsInitializer (committedOffsets),
> reword the warning and the docs to say that auto.offset.reset also decides
> where partitions without a committed offset start.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
