[jira] [Commented] (KAFKA-9020) Streams sub-topologies should be sorted by sink -> source relationship
[ https://issues.apache.org/jira/browse/KAFKA-9020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951334#comment-16951334 ] Bill Bejeck commented on KAFKA-9020: cherry-picked to 2.4 > Streams sub-topologies should be sorted by sink -> source relationship > -- > > Key: KAFKA-9020 > URL: https://issues.apache.org/jira/browse/KAFKA-9020 > Project: Kafka > Issue Type: Improvement > Components: streams >Reporter: Guozhang Wang >Assignee: Sophie Blee-Goldman >Priority: Major > Fix For: 2.4.0 > > > In Streams {{InternalTopologyBuilder}} we are sorting the sub-topologies > (named as node groups) based on the source node names: > {code} > for (final String nodeName : Utils.sorted(allSourceNodes)) { > nodeGroupId = putNodeGroupName(nodeName, nodeGroupId, nodeGroups, > rootToNodeGroup); > } > {code} > This is okay for DSL where the source node names are not customized, and > hence we can just rely on the internal suffix {{..-SOURCE-x}} which is > basically following the parsing order. But now with the source node name > customized with KIP-307, we should no longer rely on the source node names. > Instead, we should sort sub-topologies by the sink -> source relationship --- > that is, if a sub-topology A has a sink node sending to topic X while another > sub-topology B has a source node reading from X, then A should be sorted > before B. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KAFKA-9020) Streams sub-topologies should be sorted by sink -> source relationship
[ https://issues.apache.org/jira/browse/KAFKA-9020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951316#comment-16951316 ] ASF GitHub Bot commented on KAFKA-9020: --- bbejeck commented on pull request #7495: KAFKA-9020: Streams sub-topologies should be sorted by sink -> source relationship URL: https://github.com/apache/kafka/pull/7495 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Streams sub-topologies should be sorted by sink -> source relationship > -- > > Key: KAFKA-9020 > URL: https://issues.apache.org/jira/browse/KAFKA-9020 > Project: Kafka > Issue Type: Improvement > Components: streams >Reporter: Guozhang Wang >Assignee: Sophie Blee-Goldman >Priority: Major > > In Streams {{InternalTopologyBuilder}} we are sorting the sub-topologies > (named as node groups) based on the source node names: > {code} > for (final String nodeName : Utils.sorted(allSourceNodes)) { > nodeGroupId = putNodeGroupName(nodeName, nodeGroupId, nodeGroups, > rootToNodeGroup); > } > {code} > This is okay for DSL where the source node names are not customized, and > hence we can just rely on the internal suffix {{..-SOURCE-x}} which is > basically following the parsing order. But now with the source node name > customized with KIP-307, we should no longer rely on the source node names. > Instead, we should sort sub-topologies by the sink -> source relationship --- > that is, if a sub-topology A has a sink node sending to topic X while another > sub-topology B has a source node reading from X, then A should be sorted > before B. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KAFKA-9020) Streams sub-topologies should be sorted by sink -> source relationship
[ https://issues.apache.org/jira/browse/KAFKA-9020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16949071#comment-16949071 ] ASF GitHub Bot commented on KAFKA-9020: --- ableegoldman commented on pull request #7495: KAFKA-9020: Streams sub-topologies should be sorted by sink -> source relationship URL: https://github.com/apache/kafka/pull/7495 Subtopologies are currently ordered alphabetically by source node, which prior to KIP-307 happened to always result in the "correct" (ie topological) order. Now that users may name their nodes anything they want, we must explicitly order them so that upstream node groups/subtopologies come first and the downstream ones come after. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Streams sub-topologies should be sorted by sink -> source relationship > -- > > Key: KAFKA-9020 > URL: https://issues.apache.org/jira/browse/KAFKA-9020 > Project: Kafka > Issue Type: Improvement > Components: streams >Reporter: Guozhang Wang >Assignee: Sophie Blee-Goldman >Priority: Major > > In Streams {{InternalTopologyBuilder}} we are sorting the sub-topologies > (named as node groups) based on the source node names: > {code} > for (final String nodeName : Utils.sorted(allSourceNodes)) { > nodeGroupId = putNodeGroupName(nodeName, nodeGroupId, nodeGroups, > rootToNodeGroup); > } > {code} > This is okay for DSL where the source node names are not customized, and > hence we can just rely on the internal suffix {{..-SOURCE-x}} which is > basically following the parsing order. But now with the source node name > customized with KIP-307, we should no longer rely on the source node names. > Instead, we should sort sub-topologies by the sink -> source relationship --- > that is, if a sub-topology A has a sink node sending to topic X while another > sub-topology B has a source node reading from X, then A should be sorted > before B. -- This message was sent by Atlassian Jira (v8.3.4#803005)