[ https://issues.apache.org/jira/browse/KAFKA-7669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17032512#comment-17032512 ]
John Roesler commented on KAFKA-7669: ------------------------------------- This issue has been lurking for a while, and has been reported a number of different ways. It seems to take two forms: 1. changing the topology at all (in apparently compatible ways) can renumber operators and corrupt the application upon restart 2. changing the topology in combination with a rolling bounce results in members executing a different topology than the leader, which leads to extra problems (such as NPEs) https://issues.apache.org/jira/browse/KAFKA-7669 is related, and seems to be more about just changing the topology at all https://issues.apache.org/jira/browse/KAFKA-8307 proposes a fix https://issues.apache.org/jira/browse/KAFKA-8810 seems to be a duplicate of KAFKA-8307 and https://issues.apache.org/jira/browse/KAFKA-9518 reports an exception that results from a rolling-bounce topology change. > Stream topology definition is not robust to the ordering changes > ---------------------------------------------------------------- > > Key: KAFKA-7669 > URL: https://issues.apache.org/jira/browse/KAFKA-7669 > Project: Kafka > Issue Type: Wish > Components: streams > Affects Versions: 2.0.0 > Reporter: Mateusz Owczarek > Priority: Major > > It seems that if the user does not guarantee the order of the stream topology > definition, he may end up with multiple stream branches having the same > internal changelog (and repartition, if created) topic. > Let's assume: > {code:java} > val initialStream = new StreamsBuilder().stream(sth); > val someStrings = (1 to 10).map(_.toString) > val notGuaranteedOrderOfStreams: Map[String, KStream[...]] = > someStrings.map(s => s -> initialStream.filter(...)).toMap{code} > When the user defines now common aggregation logic for the > notGuaranteedOrderOfStreams, and runs multiple instances of the application > the KSTREAM-AGGREGATE-STATE-STORE topics names will not be unique and will > contain results of the different streams from notGuaranteedOrderOfStreams map. > All of this without a single warning that the topology (or just the order of > the topology definition) differs in different instances of the Kafka Streams > application. > Also, I am concerned that ids in "KSTREAM-AGGREGATE-STATE-STORE-id-changelog > " match so well for the different application instances (and different > topologies). > -- This message was sent by Atlassian Jira (v8.3.4#803005)