sanghyeok An created KAFKA-21008:
------------------------------------
Summary: Consider allowing record headers to be filtered or masked
before persistence in Kafka Streams state stores
Key: KAFKA-21008
URL: https://issues.apache.org/jira/browse/KAFKA-21008
Project: Kafka
Issue Type: Improvement
Reporter: sanghyeok An
Assignee: sanghyeok An
Kafka Streams header-aware state stores introduced by KIP-1271 can persist
record headers together with state values and propagate them through state
store changelogs.
Record headers may contain application-level sensitive information, such as
credentials, tokens, personally identifiable information, or request metadata
that should not be retained at rest.
Currently, Kafka Streams does not provide a built-in mechanism to filter,
remove, or mask selected headers before they are written to a header-aware
state store. Applications must sanitize headers manually before every stateful
operator, which is error-prone and may also modify the headers observed by
downstream processors.
Kafka Streams should provide an opt-in mechanism to filter or transform the
copy of the headers that is persisted by header-aware state stores.
The mechanism should:
* Allow selected headers to be removed or their values to be masked.
* Apply consistently to both local state and state-store changelog records.
* Avoid mutating the headers of the in-flight input record.
* Preserve the current behavior when no filtering or masking policy is
configured.
* Ensure that restored state contains the same sanitized headers that were
written to the local store.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
