[
https://issues.apache.org/jira/browse/KAFKA-21550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias J. Sax resolved KAFKA-21550.
-------------------------------------
Resolution: Fixed
> Standby task on a source-topic changelog reports end offset 0 if its
> committed offset cannot be fetched
> -------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-21550
> URL: https://issues.apache.org/jira/browse/KAFKA-21550
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 4.4.0
> Reporter: Matthias J. Sax
> Assignee: Matthias J. Sax
> Priority: Minor
> Fix For: 4.5.0, 4.4.1
>
>
> StoreChangelogReader#register() sets restoreEndOffset = 0L for a standby task
> on a source-topic changelog. This disables restoration until the committed
> offset is known.
> In initializeChangelogs(), only an active changelog is removed from
> newPartitionsToRestore when its offset lookup fails. If the committed-offset
> lookup fails for a standby, committedOffsetForChangelogs() returns an empty
> map, so updateLimitOffsetsForStandbyChangelogs() is skipped:
>
> {code:java}
> if (!committedOffsets.isEmpty()) {
> updateLimitOffsetsForStandbyChangelogs(committedOffsets);
> }{code}
>
> The changelog still moves to RESTORING with restoreEndOffset == 0.
> initializeChangelogs() only handles REGISTERED changelogs, so it never
> retries this one. The value is only corrected by
> maybeUpdateLimitOffsetsForStandbyChangelogs(), which requires
> STANDBY_UPDATING, buffered records and an elapsed commit interval. It is
> never corrected while active tasks restore.
> Meanwhile logicalChangelogEndOffsets() reports an end offset of 0 for this
> changelog. The broker computes the lag as endOffset - offset and treats the
> task as caught up.
> A standby task on a source-topic changelog should stay REGISTERED until its
> committed offset is known. Until then, its end offset should be reported as
> unknown, because logicalChangelogEndOffsets() also reports REGISTERED
> changelogs.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)