[
https://issues.apache.org/jira/browse/KAFKA-20876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Lau updated KAFKA-20876:
-----------------------------
Component/s: streams
> Windowed restore optimisation gives up when `endOffset-1` is a transaction
> control record
> -----------------------------------------------------------------------------------------
>
> Key: KAFKA-20876
> URL: https://issues.apache.org/jira/browse/KAFKA-20876
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Reporter: Alan Lau
> Assignee: Alan Lau
> Priority: Blocker
>
> When restoring a windowed store with no checkpoint, {{StoreChangelogReader}}
> seeks to {{latestTimestamp - retentionPeriod}} instead of {{{}log-start{}}},
> since data older than the retention is discarded on write. It learns
> {{latestTimestamp}} by seeking to {{endOffset - 1}} and polling once in
> {{{}seekNewPartitions{}}}; an empty poll falls back to
> {{{}seekToBeginning{}}}.
> On a 4.3 EOS-v2 soak, {{endOffset - 1}} returned no record on 195 of 195
> probes. Dumping the segment confirms the last batch is a transaction control
> record (isControl: true, count: 1, size: 78), and the pattern recurs
> throughout — every transaction ends with a marker occupying an offset that is
> never delivered to a consumer. The poll therefore returns nothing and the
> optimisation is abandoned, leaving the restore at log-start with no margin
> against retention.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
