[
https://issues.apache.org/jira/browse/KAFKA-13332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422287#comment-17422287
]
Guozhang Wang commented on KAFKA-13332:
---------------------------------------
Thanks [~ableegoldman]. My first thought on this ticket is that would be "won't
fix" in a way that, if users expect they need to create new topics with more
partitions in the future, they need to consider using
https://cwiki.apache.org/confluence/display/KAFKA/KIP-221%3A+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint
> New pattern-matched topic with more partitions than existing matched topics
> can crash Kafka Streams
> ---------------------------------------------------------------------------------------------------
>
> Key: KAFKA-13332
> URL: https://issues.apache.org/jira/browse/KAFKA-13332
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Reporter: A. Sophie Blee-Goldman
> Priority: Major
>
> The partition count resolution logic in Streams is used to determine the
> number of partitions for any repartition topics that don't already exist.
> This is done by parsing the topology to find the number of partitions of all
> upstream topics, and taking the max. For Pattern-subscribed subtopologies,
> this means you need to ensure that at least one topic matching this pattern
> is created prior to starting up the app. That topic, or topics, will
> determine the number of partitions for any downstream repartitions.
> The problem is that repartition topics are created once, the first time the
> app is started up. After that, during each rebalance Streams will validate
> all repartition topics including checking for their existence, and verifying
> they have the correct number of partitions. This check will fail if a new
> topic is created after the first initialization, which matches the pattern
> but has more partitions than any of the existing topics.
> This means that unfortunately, you can't create a new input topic that
> matches the pattern your app is subscribed to unless it has equal or fewer
> partitions than the existing matching topics. If you do, you would need to
> stop all instances and delete the existing repartition topics before creating
> this new topic
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
