[
https://issues.apache.org/jira/browse/KAFKA-21009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18111757#comment-18111757
]
Matthias J. Sax commented on KAFKA-21009:
-----------------------------------------
Overall, yes, we need this. It's the largest gap after KIP-1271 and KIP-1285
landed, that the DSL does not fully support headers. As a matter of fact, for
aggregations and joins KS still just drops all headers, because there is no
good default behavior.
When you say "policy" what do you have in mind? I mean mainly from an API POV?
It seems, instead of policies, a most straightforward (but complicated)
approach would be to allow users to implement a callback (similar to
`Initializer`, `Aggregator`, `Joiner`) that would do the header handling. – I
say "complicated" because it would be huge change on the API... We already have
secondary callbacks like `ValueJoinerWithKey` as overloads. It's not clear to
me though, how new callback should be designed.
> Consider introducing record header handling policies for Kafka Streams
> stateful operations
> ------------------------------------------------------------------------------------------
>
> Key: KAFKA-21009
> URL: https://issues.apache.org/jira/browse/KAFKA-21009
> Project: Kafka
> Issue Type: Improvement
> Reporter: sanghyeok An
> Assignee: sanghyeok An
> Priority: Minor
> Labels: needs-kip, streams
>
> Stateful operations such as joins and aggregations may combine or retain
> records from multiple inputs. Unlike one-to-one stateless operations, there
> may not be an obvious set of headers to use for the output record.
>
> Currently, header handling is determined by the behavior of each operator,
> and applications do not have an explicit policy for controlling it.
>
> We should consider whether Kafka Streams stateful operations could support
> header handling policies such as:
> * Retaining the existing behavior
> * Ignoring or dropping headers
> * Preserving headers from a selected input record
> * Merging headers from multiple input records
> This Jira is intended to capture the use case and evaluate whether such
> policies would be useful. If pursued, the affected operators, exact policy
> set, defaults, conflict handling, and consistency across caching and
> restoration should be defined through a KIP.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
