Thomas Crowley created KAFKA-8846:
-------------------------------------
Summary: Unexpected results joining a KStream to a KTable after
repartitioning
Key: KAFKA-8846
URL: https://issues.apache.org/jira/browse/KAFKA-8846
Project: Kafka
Issue Type: Bug
Components: streams
Affects Versions: 2.3.0
Reporter: Thomas Crowley
We seem to have come across a bug with Kafka Streams (or at least unexpected
behavior) when joining a KStream to a KTable after re-partitioning our data
(via `selectKey`)
Our use case is as follows: we want to aggregate some values and join them back
onto the original message, so that we emit the original message, joined with
the current value of the aggregation at the current point in time.
Currently, without re-partitioning, we get the correct behavior as expected,
but rekeying the input of the stream gives us incorrect results.
What's stranger, is that the `TestTopologyDriver` gives us the correct/expected
results in our re-partitioned topology.
Apologies if Clojure is foreign to anyone, but I have an example of the
problematic topology here:
https://github.com/VerrencyOpenSource/repartition-bug/blob/master/src/repartition_bug/with_repartition.clj#L27
If you have `lein` installed on your machine, I have instructions on how you
can run the above topology against both the test topology driver and a Kafka
cluster:
https://github.com/VerrencyOpenSource/repartition-bug
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
