[
https://issues.apache.org/jira/browse/KAFKA-21314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charles-Eddy updated KAFKA-21314:
---------------------------------
Description:
I found this while prototyping key-based ordering for share groups. It
reproduces on plain trunk.
When a share partition is at share.partition.max.record.locks,
lastOffsetAndMaxRecordsToAcquire caps lastOffsetToAcquire to the end offset. On
a compacted topic, when the group lags enough for these records to be out of
the active segment, the cleaner may remove the records between the fetch offset
and the end offset, so the first fetched batch can start after the end offset,
and acquire() calls cachedState.subMap(from, to) with from > to:
{code:java}
java.lang.IllegalArgumentException: inconsistent range
at
java.base/java.util.concurrent.ConcurrentSkipListMap.subMap(ConcurrentSkipListMap.java:1912)
at kafka.server.share.SharePartition.acquire(SharePartition.java:842){code}
In the simple case maybeArchiveStaleBatches has already archived the missing
offsets, so one ShareFetch request fails with UNKNOWN_SERVER_ERROR and the
partition recovers. After a re-initialization with a persister read gap at the
start of the window, the gap window does not advance and every fetch at the gap
start fails the same way (I checked 3 consecutive fetches).
The attached patch adds a test for the simple
case:SharePartitionTest#testAcquireAtMaxInFlightRecordsWhenAvailableRecordsAreCompactedAway.
Returning an empty result when the first fetched batch starts after
lastOffsetToAcquire, and moving the gap start past the compacted range, should
fix it, I don't know if it creates other issues though, I let you decide how to
solve this.
was:
I found this while prototyping key-based ordering for share groups. It
reproduces on plain trunk.
When a share partition is at share.partition.max.record.locks,
lastOffsetAndMaxRecordsToAcquire caps lastOffsetToAcquire to the end offset. On
a compacted topic the records between the fetch offset and the end offset may
be gone, so the first fetched batch can start after the end offset, and
acquire() calls cachedState.subMap(from, to) with from > to:
{code:java}
java.lang.IllegalArgumentException: inconsistent range
at
java.base/java.util.concurrent.ConcurrentSkipListMap.subMap(ConcurrentSkipListMap.java:1912)
at kafka.server.share.SharePartition.acquire(SharePartition.java:842){code}
In the simple case maybeArchiveStaleBatches has already archived the missing
offsets, so one ShareFetch request fails with UNKNOWN_SERVER_ERROR and the
partition recovers. After a re-initialization with a persister read gap at the
start of the window, the gap window does not advance and every fetch at the gap
start fails the same way (I checked 3 consecutive fetches).
The attached patch adds a test for the simple
case:SharePartitionTest#testAcquireAtMaxInFlightRecordsWhenAvailableRecordsAreCompactedAway.
Returning an empty result when the first fetched batch starts after
lastOffsetToAcquire, and moving the gap start past the compacted range, should
fix it, I don't know if it creates other issues though, I let you decide how to
solve this.
> Share partition acquire throws "inconsistent range" at the in-flight limit on
> a compacted topic
> -----------------------------------------------------------------------------------------------
>
> Key: KAFKA-21314
> URL: https://issues.apache.org/jira/browse/KAFKA-21314
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 4.5.0
> Reporter: Charles-Eddy
> Assignee: Apoorv Mittal
> Priority: Minor
> Attachments: 2-inconsistent-range-at-in-flight-limit.patch
>
>
> I found this while prototyping key-based ordering for share groups. It
> reproduces on plain trunk.
> When a share partition is at share.partition.max.record.locks,
> lastOffsetAndMaxRecordsToAcquire caps lastOffsetToAcquire to the end offset.
> On a compacted topic, when the group lags enough for these records to be out
> of the active segment, the cleaner may remove the records between the fetch
> offset and the end offset, so the first fetched batch can start after the end
> offset, and acquire() calls cachedState.subMap(from, to) with from > to:
> {code:java}
> java.lang.IllegalArgumentException: inconsistent range
> at
> java.base/java.util.concurrent.ConcurrentSkipListMap.subMap(ConcurrentSkipListMap.java:1912)
> at
> kafka.server.share.SharePartition.acquire(SharePartition.java:842){code}
> In the simple case maybeArchiveStaleBatches has already archived the missing
> offsets, so one ShareFetch request fails with UNKNOWN_SERVER_ERROR and the
> partition recovers. After a re-initialization with a persister read gap at
> the start of the window, the gap window does not advance and every fetch at
> the gap start fails the same way (I checked 3 consecutive fetches).
> The attached patch adds a test for the simple
> case:SharePartitionTest#testAcquireAtMaxInFlightRecordsWhenAvailableRecordsAreCompactedAway.
> Returning an empty result when the first fetched batch starts after
> lastOffsetToAcquire, and moving the gap start past the compacted range,
> should fix it, I don't know if it creates other issues though, I let you
> decide how to solve this.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
