[ 
https://issues.apache.org/jira/browse/KAFKA-21550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matthias J. Sax resolved KAFKA-21550.
-------------------------------------
    Resolution: Fixed

> Standby task on a source-topic changelog reports end offset 0 if its 
> committed offset cannot be fetched
> -------------------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-21550
>                 URL: https://issues.apache.org/jira/browse/KAFKA-21550
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>    Affects Versions: 4.4.0
>            Reporter: Matthias J. Sax
>            Assignee: Matthias J. Sax
>            Priority: Minor
>             Fix For: 4.5.0, 4.4.1
>
>
> StoreChangelogReader#register() sets restoreEndOffset = 0L for a standby task 
> on a source-topic changelog. This disables restoration until the committed 
> offset is known.
> In initializeChangelogs(), only an active changelog is removed from 
> newPartitionsToRestore when its offset lookup fails. If the committed-offset 
> lookup fails for a standby, committedOffsetForChangelogs() returns an empty 
> map, so updateLimitOffsetsForStandbyChangelogs() is skipped:
>  
> {code:java}
> if (!committedOffsets.isEmpty()) {
>     updateLimitOffsetsForStandbyChangelogs(committedOffsets);
> }{code}
>  
> The changelog still moves to RESTORING with restoreEndOffset == 0. 
> initializeChangelogs() only handles REGISTERED changelogs, so it never 
> retries this one. The value is only corrected by 
> maybeUpdateLimitOffsetsForStandbyChangelogs(), which requires 
> STANDBY_UPDATING, buffered records and an elapsed commit interval. It is 
> never corrected while active tasks restore.
> Meanwhile logicalChangelogEndOffsets() reports an end offset of 0 for this 
> changelog. The broker computes the lag as endOffset - offset and treats the 
> task as caught up.
> A standby task on a source-topic changelog should stay REGISTERED until its 
> committed offset is known. Until then, its end offset should be reported as 
> unknown, because logicalChangelogEndOffsets() also reports REGISTERED 
> changelogs.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to