muralibasani commented on code in PR #22458: URL: https://github.com/apache/kafka/pull/22458#discussion_r4383095733
########## docs/streams/upgrade-guide.md: ########## @@ -97,6 +97,8 @@ As part of [KIP-1284](https://cwiki.apache.org/confluence/spaces/KAFKA/pages/406 [KIP-1356](https://cwiki.apache.org/confluence/spaces/KAFKA/pages/430408653/KIP-1356+Introduce+IQv2+for+headers-aware+state+stores) adds four `@Evolving` IQv2 query types that return record headers from the [header-aware state stores](#kip-1271-headers-aware-stores) introduced by KIP-1271: `TimestampedKeyWithHeadersQuery`, `TimestampedRangeWithHeadersQuery`, `TimestampedWindowKeyWithHeadersQuery`, and `TimestampedWindowRangeWithHeadersQuery`. Their results carry headers as a [`ReadOnlyRecord`](/{version}/javadoc/org/apache/kafka/streams/processor/api/ReadOnlyRecord.html) — or, for the range and window forms, a closeable [`ReadOnlyRecordIterator`](/{version}/javadoc/org/apache/kafka/streams/state/ReadOnlyRecordIterator.html). See the [interactive queries guide](/{version}/streams/developer-guide/interactive-queries/#header-aware-stores-interactive-queries) for usage and behavior details. +[KIP-770](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=186878390) is now fully implemented: config `input.buffer.max.bytes` (LONG, default 512 MB, topology-wide) is introduced and `buffered.records.per.partition` is deprecated ([KAFKA-13152](https://issues.apache.org/jira/browse/KAFKA-13152)). The total buffer is split evenly across stream threads (`input.buffer.max.bytes / num.stream.threads`); when a thread's tasks collectively exceed their slice, the main consumer pauses every non-empty input partition on that thread and resumes once the buffered bytes drop back below the cap. When `input.buffer.max.bytes` is set, the new bytes-based topology-wide pause supersedes the legacy per-partition record-count pause. If only the deprecated `buffered.records.per.partition` is set, the bytes cap is still applied at its default (512 MB) so memory stays bounded, while the legacy per-partition record cap continues to apply in parallel; a deprecation warning is logged. Memor y note: upgraders who never set `input.buffer.max.bytes` will reserve 512 MB for input buffering by default — set it explicitly on memory-constrained deployments. Review Comment: Nice ones. Fixed both. It now says input.buffer.max.bytes is an upper bound, not an up-front allocation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
