apoorvmittal10 opened a new pull request, #21051: URL: https://github.com/apache/kafka/pull/21051
The PR fixes the issue when SharePartition is re-initalized and starts fetch prior to start offset. In edge cases there occurs a scenario when Acquire method creates overlapping batches in cache. The issue is more evident in `record_limit` mode. Consider a scenario where log has 0-99 offsets as batch and post re-initialization of share partition the start offset becomes 5, first 5 records already acknowledged in previous share-partition instance. Prior to the fix: - The request arrives for fetching next 5 offsets hence the cache will hold 5-99, with 5-9 offsets as acquired. - Client acknowledes 5-9 offset, start offset moves to 10. - Client re-fetches from offset 10 and gets same 0-99 batch. - Acquire readjusts the base offset to 10 as start offset has moved. - There won't be any overlapping batch in cache for 10-99 as the key in cache is 5. - Hence, 0-99 cache entry will be created Post Fix: - There will be an overlapping entry in the cache as start considering the minimum of baseOffset and batch's firstOffset. Also fixed other scenarios where due to startOffset move and no cache overlap the offsets could be acquired prior to startOffset. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
