hzh0425 opened a new pull request, #15141: URL: https://github.com/apache/kafka/pull/15141
### Problem https://issues.apache.org/jira/browse/KAFKA-16073 The consumer will receive 'OutOfRangeException' in the following case: In a specific concurrent scenario, imagine sequential offsets: offset1 < offset2 < offset3. A client requests data at offset2. While a background deletion process removes segments from memory, it hasn't yet updated the LocalLogStartOffset from offset1 to offset3. Consequently, when the fetch offset (offset2) is evaluated against the stale offset1 in ReplicaManager.handleOffsetOutOfRangeError, it incorrectly triggers an OffsetOutOfRangeException. This issue arises from the out-of-sync update of localLogStartOffset, leading to incorrect handling of consumer fetch requests and potential data access errors. Let us assume each segment has one offset in this example: log start offset 0 log end offset 10 local log start offset 4 fetch offset 6 new local log start offset 7 Deletion based on retention configs is started and eventually updating the local log start offset as 7. There is a race condition here where the segments list is updated by removing 4, 5, and 6 offset segments in LocalLog and then updates the local-log-start-offset. But fetch offset is being served concurrently and it may throw OffsetOutOfRangeException if the inmemory segments are already removed in LocalLog and local-log-start-offset is not yet updated as 7 when it executes the [code](https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/server/ReplicaManager.scala#L1866) as it fails the condition because fetch offset(6) < old local-log-start-offset(4). ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
