Charles-Eddy created KAFKA-21315:
------------------------------------
Summary: Share partition stops delivering records after an
acknowledgement is rolled back below a moved start offset
Key: KAFKA-21315
URL: https://issues.apache.org/jira/browse/KAFKA-21315
Project: Kafka
Issue Type: Bug
Components: core
Affects Versions: 4.5.0
Reporter: Charles-Eddy
Attachments: 4-partition-stall-after-rolled-back-ack.patch
I found this while prototyping key-based ordering for share groups. It
reproduces on plain trunk.
If an ACCEPT is being written to the persister when the log start offset moves
past the acknowledged batch (retention or DeleteRecords catching up with a
lagging group), archiveRecords leaves the batch alone because it is not
AVAILABLE. If its acquisition lock then expires and the write fails, for
example while the share coordinator is unavailable,
completeStateTransition(false) moves the batch back to AVAILABLE, now below the
start offset.
>From then on nextFetchOffset() returns that offset, the fetch fails with
>OFFSET_OUT_OF_RANGE, and updateCacheAndOffsets() returns early because the log
>start offset is not greater than the start offset. Every fetch logs "The log
>start offset: 6 is not greater than the start offset: 6 ..." and returns no
>records, so the records after the start offset are never delivered. Only
>re-creating the SharePartition (leader change or broker restart) seems to
>recover it.
The attached patch adds a test that reproduces
it:SharePartitionTest#testAcceptRolledBackAfterLockExpiryBelowMovedStartOffsetDoesNotStallFetches
(nextFetchOffset() returns 0 instead of 6).
The lock timeout path already archives records below the start offset. Doing
the same when a transition is rolled back with an expired lock should fix it,
and nextFetchOffset() could also skip offsets below the start offset. I let you
decide what is the best way to handle that though.
Thanks
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
