Pascal Büttiker created KAFKA-8917:
--------------------------------------
Summary: When performing a Left/Right-Join, pick the headers of
the same side
Key: KAFKA-8917
URL: https://issues.apache.org/jira/browse/KAFKA-8917
Project: Kafka
Issue Type: Improvement
Components: streams
Reporter: Pascal Büttiker
As described in KAFKA-7718, headers are promoted by the "triggering" record in
stateful operations such as Joins. This was very confusing and we spent quite
some time debugging this.
While we ideally have full control over this behaviour as like the KAFKA-7718
proposes, I hope we can solve some of the randomness before this:
* Inner-Join: Keep as is (use the headers of the triggering record)
* Full-Join: Keep as is (use the headers of the triggering record)
* Left-Join: *Always pick the headers of the left record.*
* Right-Join: *Always pick the headers of the right record.*
This behaviour would solve the most pressing issues when dealing with headers
in Kafka Streams.
*Motivation*:
We depend on headers, especially when dealing with tombstone records. There is
no other way to store additional information. If we do not use tombstone
records, all default Kafka Features around compacted topics and KTabels are no
longer useable. We are able to use custom Transformers to generate the headers
(basically patch in the missing header support in Kafka Streams), but as soon
that we use Join/Aggregate we loose control over the headers.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
