unknowntpo opened a new pull request, #23815: URL: https://github.com/apache/kafka/pull/23815
A standby task whose store uses the input topic as its changelog may only apply records up to the offset committed by the active task, but `StoreChangelogReader` kept fetching beyond that offset. Those records stayed in the restore buffer, which could grow until `OutOfMemoryError` when the active task was far behind the end of the topic. This PR pauses such a changelog partition while its buffer holds records beyond the limit offset, and resumes it once the limit has moved past them. This is how standby tasks worked before 2.5.0: `StreamThread` paused the partition when `StandbyTask#update` returned records beyond the limit, and this was lost in the refactoring of KAFKA-9113 (#7997). The change adds no new config or public API. The new integration test pauses the active task and produces 10,000 records: without the fix the standby polls all 10,000 records beyond the committed offset; with the fix it polls them once (at most `max.poll.records`) and then stops. Related to KAFKA-19992 / KIP-1325. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
