AndrewJSchofield opened a new pull request, #23499: URL: https://github.com/apache/kafka/pull/23499
We have unearthed a memory leak in the share consumer. It appears to be related to merging of instances of `ShareInFlightBatch` when we have multiple batches for a single topic-partition awaiting delivery to the application code. The prior code merged batches by adding the records from a later batch to the existing batch. However, the merging operation was not really sufficiently sophisticated to handle the various different situations in which a share consumer can find itself. This PR removes the merging of batches and moves to a list of batches per topic-partition instead. This is essentially a refactor of `ShareFetch`. All existing client tests pass unmodified. A handful of new tests have been added to test the specific situations in which merging was occurring. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
