[
https://issues.apache.org/jira/browse/KAFKA-13024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Damien Gasparina updated KAFKA-13024:
-------------------------------------
Description:
{{KStream.repartition}} is silently filtering messages with null keys. A single
topology like {{.stream().repartition().to()}} would filter all messages with
null key.
The cause: we are adding a filter before the source & sink nodes
([https://github.com/apache/kafka/blob/2.8/streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamImpl.java#L1060-L1064]).
It looks like we are doing that because this method is also used for building
KTable.
Null key messages are valid for a KStream, it looks like a regression, the
previous {{.through()}} was not filtering null key messages.
was:
{{KStream.repartition}} is silently filtering messages with null keys. A single
topology like {{.stream().repartition().to()}} would filter all messages with
null key.
We are adding a filtering before the repartition
([https://github.com/apache/kafka/blob/2.8/streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamImpl.java#L1060-L1064]).
It looks like we are doing that because this method is also used for building
KTable.
Null key messages are valid for a KStream, it looks like a regression, the
previous {{.through()}} was not filtering null key messages.
> Kafka Streams is dropping messages with null key during repartition
> -------------------------------------------------------------------
>
> Key: KAFKA-13024
> URL: https://issues.apache.org/jira/browse/KAFKA-13024
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 2.8.0, 2.7.1
> Reporter: Damien Gasparina
> Priority: Major
>
> {{KStream.repartition}} is silently filtering messages with null keys. A
> single topology like {{.stream().repartition().to()}} would filter all
> messages with null key.
> The cause: we are adding a filter before the source & sink nodes
> ([https://github.com/apache/kafka/blob/2.8/streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamImpl.java#L1060-L1064]).
> It looks like we are doing that because this method is also used for
> building KTable.
> Null key messages are valid for a KStream, it looks like a regression, the
> previous {{.through()}} was not filtering null key messages.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
