This is an automated email from the ASF dual-hosted git repository.
mjsax pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/kafka.git
The following commit(s) were added to refs/heads/trunk by this push:
new e7cba22471f KAFKA-20849: Update the headers-aware store docs on
suppress() (#23007)
e7cba22471f is described below
commit e7cba22471f0d2833b850e2b0085c1cfff2efed5
Author: Alieh Saeedi <[email protected]>
AuthorDate: Sat Aug 1 02:09:41 2026 +0200
KAFKA-20849: Update the headers-aware store docs on suppress() (#23007)
KAFKA-20413 made the `suppress()` buffer headers-aware when
`dsl.store.format=HEADERS` is configured
(`InMemoryTimeOrderedKeyValueChangeBuffer`), but four
documentation locations still claim that `suppress()` loses headers:
- `dsl-api.md` "Note on headers-aware state stores" on `suppress()`
- `dsl-api.md` KIP-1285 current-behavior bullets
- `config-streams.md` `dsl.store.format` "Current limitations"
- `upgrade-guide.md` KIP-1285 "Current limitations"
Docs only -- no code changes.
Reviewers: Matthias J. Sax <[email protected]>
---
docs/streams/developer-guide/config-streams.md | 2 +-
docs/streams/developer-guide/dsl-api.md | 4 +---
docs/streams/upgrade-guide.md | 2 ++
3 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/docs/streams/developer-guide/config-streams.md
b/docs/streams/developer-guide/config-streams.md
index 5c006ddaf8a..ec31bb4222c 100644
--- a/docs/streams/developer-guide/config-streams.md
+++ b/docs/streams/developer-guide/config-streams.md
@@ -1407,7 +1407,7 @@ Serde for the inner class of a windowed record. Must
implement the `Serde` inter
>
> See [KIP-1271](https://cwiki.apache.org/confluence/x/QIM8G) for migration
> procedures, changelog compatibility, restore behavior, and per-record
> overhead.
>
-> **Current limitations**: `dsl.store.format=HEADERS` changes the state store
format. It does not define how DSL operators create headers for output records.
Some operators write empty headers to their materialized stores, and the buffer
stores used by `suppress()` and left/outer stream-stream joins are not
headers-aware. See [Stateful
transformations](/{version}/streams/developer-guide/dsl-api.html#stateful-transformations)
and the [Streams upgrade guide](/{version}/streams/upgrade-guid [...]
+> **Current limitations**: `dsl.store.format=HEADERS` changes the state store
format. It does not define how DSL operators create headers for output records.
Some operators write empty headers to their materialized stores, and the buffer
store used by left/outer stream-stream joins for not-yet-matched records is not
headers-aware. See [Stateful
transformations](/{version}/streams/developer-guide/dsl-api.html#stateful-transformations)
and the [Streams upgrade guide](/{version}/streams/upg [...]
### ensure.explicit.internal.resource.naming
diff --git a/docs/streams/developer-guide/dsl-api.md
b/docs/streams/developer-guide/dsl-api.md
index 6ed4e375784..34f18db6e70 100644
--- a/docs/streams/developer-guide/dsl-api.md
+++ b/docs/streams/developer-guide/dsl-api.md
@@ -931,7 +931,7 @@ This config only changes the state store format. It does
not define how DSL oper
* aggregations, KTable-KTable joins, materialized `KTable.mapValues`,
`KStream.toTable()`, and `StreamsBuilder.table()` write empty headers to their
materialized stores
* KStream-KStream join window stores keep source-record headers, but join
result records do not get computed or merged headers; they may carry the
headers from the record that triggered the result
- * `suppress()` and left/outer [KStream-KStream joins](#kstream-kstream-join)
use non-headers-aware buffer stores, so records passing through those buffers
lose their headers
+ * left/outer [KStream-KStream joins](#kstream-kstream-join) use a
non-headers-aware buffer store for not-yet-matched records, so records passing
through that buffer lose their headers
A follow-up KIP will define how DSL result headers are computed.
@@ -4673,8 +4673,6 @@ The key parts of this program are:
One thing to note is that suppression is just like any other Kafka Streams
operator, so you can build a topology with two branches emerging from the
`count`, one suppressed, and one not, or even multiple differently configured
suppressions. This allows you to apply suppressions where they are needed and
otherwise rely on the default continuous update behavior.
-**Note on headers-aware state stores:** `suppress()` uses an in-memory buffer
that is not headers-aware. Record headers attached to upstream records are not
preserved across the suppression boundary, even when
[`dsl.store.format=HEADERS`](../config-streams#dsl-store-format) is set
globally per [KIP-1285](https://cwiki.apache.org/confluence/x/4ow8G).
-
For more detailed information, see the JavaDoc on the `Suppressed` config
object and [KIP-328](https://cwiki.apache.org/confluence/x/sQU0BQ "KIP-328").
Applying processors (Processor API integration)
diff --git a/docs/streams/upgrade-guide.md b/docs/streams/upgrade-guide.md
index fa6222fc55d..b4bd4c90482 100644
--- a/docs/streams/upgrade-guide.md
+++ b/docs/streams/upgrade-guide.md
@@ -77,6 +77,8 @@ For applications using the Streams Rebalance Protocol
(`group.protocol=streams`)
For applications using the Streams Rebalance Protocol
(`group.protocol=streams`), the broker-side task assignor is now pluggable
([KIP-1357](https://cwiki.apache.org/confluence/x/NoSnGQ)). Operators register
assignors with the new broker configuration `group.streams.assignors`, which
accepts a list of built-in assignor names and fully qualified class names of
custom `TaskAssignor` implementations; the first entry is the default for
groups that do not select one. An individual group picks [...]
+The buffer used by `suppress()` is now headers-aware, closing one of the gaps
listed under the [KIP-1285 current limitations](#current-limitations) in 4.3.0.
With
[`dsl.store.format=HEADERS`](/{version}/streams/developer-guide/config-streams.html#dsl-store-format),
each buffered value is stored together with the headers of the record it came
from, so record headers are preserved across the suppression boundary: the
record emitted when a buffered row is evicted carries the headers of the [...]
+
## Streams API changes in 4.3.0
**Note:** Kafka Streams 4.3.0 contains a critical native memory leak in the
RocksDB state store layer
([KAFKA-20616](https://issues.apache.org/jira/browse/KAFKA-20616)). The
`ColumnFamilyOptions` for the offsets column family is not closed, and column
family handles can leak on close-path exceptions, which under cascading task
closes (e.g., rebalances or error-triggered recoveries) leads to unbounded
off-heap memory growth and eventual OOM. Users running Kafka Streams should
consider upg [...]
