[
https://issues.apache.org/jira/browse/KAFKA-21314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apoorv Mittal reassigned KAFKA-21314:
-------------------------------------
Assignee: Apoorv Mittal
> Share partition acquire throws "inconsistent range" at the in-flight limit on
> a compacted topic
> -----------------------------------------------------------------------------------------------
>
> Key: KAFKA-21314
> URL: https://issues.apache.org/jira/browse/KAFKA-21314
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 4.5.0
> Reporter: Charles-Eddy
> Assignee: Apoorv Mittal
> Priority: Minor
> Attachments: 2-inconsistent-range-at-in-flight-limit.patch
>
>
> I found this while prototyping key-based ordering for share groups. It
> reproduces on plain trunk.
> When a share partition is at share.partition.max.record.locks,
> lastOffsetAndMaxRecordsToAcquire caps lastOffsetToAcquire to the end offset.
> On a compacted topic the records between the fetch offset and the end offset
> may be gone, so the first fetched batch can start after the end offset, and
> acquire() calls cachedState.subMap(from, to) with from > to:
> {code:java}
> java.lang.IllegalArgumentException: inconsistent range
> at
> java.base/java.util.concurrent.ConcurrentSkipListMap.subMap(ConcurrentSkipListMap.java:1912)
> at
> kafka.server.share.SharePartition.acquire(SharePartition.java:842){code}
> In the simple case maybeArchiveStaleBatches has already archived the missing
> offsets, so one ShareFetch request fails with UNKNOWN_SERVER_ERROR and the
> partition recovers. After a re-initialization with a persister read gap at
> the start of the window, the gap window does not advance and every fetch at
> the gap start fails the same way (I checked 3 consecutive fetches).
> The attached patch adds a test for the simple
> case:SharePartitionTest#testAcquireAtMaxInFlightRecordsWhenAvailableRecordsAreCompactedAway.
> Returning an empty result when the first fetched batch starts after
> lastOffsetToAcquire, and moving the gap start past the compacted range,
> should fix it, I don't know if it creates other issues though, I let you
> decide how to solve this.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
