[
https://issues.apache.org/jira/browse/KAFKA-6328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16301859#comment-16301859
]
ASF GitHub Bot commented on KAFKA-6328:
---------------------------------------
guozhangwang commented on issue #4339: [KAFKA-6328] Exclude node groups
belonging to global stores in InternalTopologyBuilder#makeNodeGroups
URL: https://github.com/apache/kafka/pull/4339#issuecomment-353661424
@ConcurrencyPractitioner The problem that I was pointing out, is that we
need to keep compatibility in our mind: assume you are going to upgrade your
streams app from 1.0.0 to 1.1.0 with this fix, and then task `3_3` becomes task
`2_3` because we reorder the sub-topology ids, then sticky assignor will not
work and we would unnecessarily move tasks around which would add significant
rebalance delay. So as long as your proposed solution can still maintain the
indexing that would be fine, though I personally cannot come up with a better
solution that would keep compatibility.
Happy holidays! We can chat later when you are back.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Exclude node groups belonging to global stores in
> InternalTopologyBuilder#makeNodeGroups
> ----------------------------------------------------------------------------------------
>
> Key: KAFKA-6328
> URL: https://issues.apache.org/jira/browse/KAFKA-6328
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 1.0.0
> Reporter: Guozhang Wang
> Assignee: Richard Yu
> Labels: newbie
> Attachments: kafka-6328.diff
>
>
> Today when we group processor nodes into groups (i.e. sub-topologies), we
> assign the sub-topology id for global tables' dummy groups as well. As a
> result, the subtopology ids (and hence task ids) are not consecutive anymore.
> This is quite confusing for users trouble shooting and debugging; in
> addition, the node group for global stores are not useful as well: we simply
> exclude it in all the caller functions of makeNodeGroups.
> It would be better to simply exclude the global store's node groups in this
> function so that the subtopology ids and task ids are consecutive.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
