[
https://issues.apache.org/jira/browse/KAFKA-21315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apoorv Mittal reassigned KAFKA-21315:
-------------------------------------
Assignee: Apoorv Mittal
> Share partition stops delivering records after an acknowledgement is rolled
> back below a moved start offset
> -----------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-21315
> URL: https://issues.apache.org/jira/browse/KAFKA-21315
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 4.5.0
> Reporter: Charles-Eddy
> Assignee: Apoorv Mittal
> Priority: Major
> Attachments: 4-partition-stall-after-rolled-back-ack.patch
>
>
> I found this while prototyping key-based ordering for share groups. It
> reproduces on plain trunk.
> If an ACCEPT is being written to the persister when the log start offset
> moves past the acknowledged batch (retention or DeleteRecords catching up
> with a lagging group), archiveRecords leaves the batch alone because it is
> not AVAILABLE. If its acquisition lock then expires and the write fails, for
> example while the share coordinator is unavailable,
> completeStateTransition(false) moves the batch back to AVAILABLE, now below
> the start offset.
> From then on nextFetchOffset() returns that offset, the fetch fails with
> OFFSET_OUT_OF_RANGE, and updateCacheAndOffsets() returns early because the
> log start offset is not greater than the start offset. Every fetch logs "The
> log start offset: 6 is not greater than the start offset: 6 ..." and returns
> no records, so the records after the start offset are never delivered. Only
> re-creating the SharePartition (leader change or broker restart) seems to
> recover it.
> The attached patch adds a test that reproduces
> it:SharePartitionTest#testAcceptRolledBackAfterLockExpiryBelowMovedStartOffsetDoesNotStallFetches
> (nextFetchOffset() returns 0 instead of 6).
> The lock timeout path already archives records below the start offset. Doing
> the same when a transition is rolled back with an expired lock should fix it,
> and nextFetchOffset() could also skip offsets below the start offset. I let
> you decide what is the best way to handle that though.
>
> Thanks
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
