[
https://issues.apache.org/jira/browse/KAFKA-21319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charles-Eddy updated KAFKA-21319:
---------------------------------
Priority: Minor (was: Major)
> Share partition read_committed acquire throws "Batch record not found" when
> the start offset is inside an aborted transaction batch
> -----------------------------------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-21319
> URL: https://issues.apache.org/jira/browse/KAFKA-21319
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 4.5.0
> Reporter: Charles-Eddy
> Priority: Minor
> Attachments: 3-batch-record-not-found-read-committed.patch
>
>
> I found this while prototyping key-based ordering for share groups. It
> reproduces on plain trunk.
> With share.isolation.level=read_committed,
> filterAbortedTransactionalAcquiredRecords looks up each aborted batch of the
> fetched data with cachedState.floorEntry(batch.baseOffset()). If the share
> partition start offset lies inside an aborted batch, for example after
> DeleteRecords to a mid-batch offset or, from reading the code, when a
> by_duration auto offset reset lands there, the cached batch starts at the
> start offset, so nothing in the cache starts at or below the aborted batch
> base offset, and fetchSubMap throws after the records were acquired:
> {code:java}
> java.lang.IllegalStateException: Batch record not found. The request batch
> offsets are not found in the cache.
> at kafka.server.share.SharePartition.fetchSubMap(SharePartition.java:3309)
> at
> kafka.server.share.SharePartition.filterAbortedTransactionalAcquiredRecords(SharePartition.java:3207)
> at
> kafka.server.share.SharePartition.acquire(SharePartition.java:861){code}
> The fetched records stay ACQUIRED and are not delivered, and each retry after
> the lock expires fails the same way and uses a delivery attempt.
> With share.delivery.count.limit=2, the committed records after the aborted
> batch are archived without ever being delivered. With the default limit of 5,
> the committed records are delivered in the end, after 7 failed fetches, once
> delivery throttling has archived the aborted records one by one.
> The attached patch adds a test that reproduces it:
> SharePartitionTest#testAcquireReadCommittedWhenStartOffsetIsWithinAbortedTransactionBatch.
> Looking up max(batch.baseOffset(), startOffset) instead of
> batch.baseOffset(), or skipping aborted batches that do not overlap the
> cached range, should fix it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
