apoorvmittal10 opened a new pull request, #23911: URL: https://github.com/apache/kafka/pull/23911
DelayedShareFetch estimates the available bytes for minBytes by comparing the end offset with the cached LogOffsetMetadata of the fetch offset. The cache was kept on SharePartition, so it outlived the request which populated it. A partition that is read but trimmed from the response, or whose request fails, kept a valid cache entry although its fetch offset did not advance, which requires resets and made the read path depend on state left behind by earlier requests. Keep the metadata inside DelayedShareFetch for the lifetime of the request, as DelayedFetch does with FetchPartitionStatus. The retries of tryComplete in purgatory still avoid reading from the log, and the cache can no longer go stale across requests, so the remote fetch reset is not needed. Add ShareConsumerFairnessTest to verify that two partitions with a deep backlog are both served under record_limit acquire mode. There are also cases where if a record is released where it was earlier fetch then request has to go through timeout to fetch data. Those scenarios are also handled by this fix. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
