[
https://issues.apache.org/jira/browse/KAFKA-20876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18101111#comment-18101111
]
Yunseop Eom commented on KAFKA-20876:
-------------------------------------
Verification update for PR #23029: https://github.com/apache/kafka/pull/23029
The full :streams:test suite completed successfully in 6m 54s. The targeted
StoreChangelogReaderTest, affected streams checkstyle and SpotBugs tasks, and
git diff --check also pass. The branch is clean and the PR is ready for
maintainer review.
> Windowed restore optimisation gives up when `endOffset-1` is a transaction
> control record
> -----------------------------------------------------------------------------------------
>
> Key: KAFKA-20876
> URL: https://issues.apache.org/jira/browse/KAFKA-20876
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 4.4.0
> Reporter: Alan Lau
> Assignee: Alan Lau
> Priority: Blocker
>
> When restoring a windowed store with no checkpoint, {{StoreChangelogReader}}
> seeks to {{latestTimestamp - retentionPeriod}} instead of {{{}log-start{}}},
> since data older than the retention is discarded on write. It learns
> {{latestTimestamp}} by seeking to {{endOffset - 1}} and polling once in
> {{{}seekNewPartitions{}}}; an empty poll falls back to
> {{{}seekToBeginning{}}}.
> On a 4.3 EOS-v2 soak, {{endOffset - 1}} returned no record on 195 of 195
> probes. Dumping the segment confirms the last batch is a transaction control
> record (isControl: true, count: 1, size: 78), and the pattern recurs
> throughout — every transaction ends with a marker occupying an offset that is
> never delivered to a consumer. The poll therefore returns nothing and the
> optimisation is abandoned, leaving the restore at log-start with no margin
> against retention.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
