[
https://issues.apache.org/jira/browse/KAFKA-21159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bill Bejeck reassigned KAFKA-21159:
-----------------------------------
Assignee: Bill Bejeck
> Clear next offsets to consume on task close to avoid stale commit after
> revive
> -------------------------------------------------------------------------------
>
> Key: KAFKA-21159
> URL: https://issues.apache.org/jira/browse/KAFKA-21159
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Reporter: Bill Bejeck
> Assignee: Bill Bejeck
> Priority: Major
> Fix For: 4.5.0, 4.4.1
>
>
> A revived active task (`closeDirtyAndRevive`) kept
> `nextOffsetsAndMetadataToBeConsumed`
> from before the corruption. After the consumer is seeked back to its
> committed offset, a
> commit driven by processor metadata (`needsCommit`, which commits all input
> partitions)
> could fall back to that stale next offset and commit past records the task
> hasn't
> reprocessed — skipping them if the task later moves, and purging them for a
> repartition
> topic. The KAFKA-20416 `consumedOffsets.clear()` fix does not cover this path.
> Fix: clear `nextOffsetsAndMetadataToBeConsumed` in `StreamTask.close()`,
> alongside
> `consumedOffsets`, so a revived task has no stale fallback for a partition it
> hasn't
> re-read.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
