muralibasani opened a new pull request, #22458: URL: https://github.com/apache/kafka/pull/22458
Ref : https://issues.apache.org/jira/browse/KAFKA-13152 This PR continues PR #20292 (which became inactive). It includes the same set of changes plus the gap fixes listed below. Summary - Implements the input-buffer half of KIP-770: introduces `input.buffer.max.bytes` (LONG, default 512 mb) and deprecates `buffered.records.per.partition`. (The cache half is already shipped in 3.4 (`statestore.cache.max.bytes`).) - Tracks raw bytes per `RecordQueue` and aggregates at `PartitionGroup`/`StreamTask`; `StreamThread.pollPhase` pauses non-empty partitions when per-thread allocation is exceeded; resumes once buffered bytes fall back below the cap. - Adds DEBUG task-level metric `input-buffer-bytes-total`. - Reallocates per-thread cache and buffer slices together on `addStreamThread`/`removeStreamThread`. - Closes gaps that PR #20292 did not address: (CI failures, corrupted-record bytes-counter inflation in `RecordQueue.updateHead`, doc sync, trunk rebase). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
