[
https://issues.apache.org/jira/browse/KAFKA-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15955544#comment-15955544
]
Matthias J. Sax edited comment on KAFKA-5009 at 4/4/17 6:16 PM:
----------------------------------------------------------------
All topics generated by Kafka Streams are prefixed with {{application.id}} --
thus, multiple Kafka Streams instances should not conflict on topic names --
topic names should be globally unique already. Can you explain in more detail
what conflicts you did observe? I am also not sure, who this would related to
Confluent Schema Registry?
was (Author: mjsax):
All topics generated by Kafka Streams are prefixed with {{application.id}} --
thus, multiple Kafka Streams instances should not conflict on topic names. Can
you explain in more detail what conflicts you did observe? I am also not sure,
who this would related to Confluent Schema Registry?
> Globally Unique internal topic names when using Joins
> ------------------------------------------------------
>
> Key: KAFKA-5009
> URL: https://issues.apache.org/jira/browse/KAFKA-5009
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Mark Tranter
>
> We are using multiple different Kafka Streams applications on the back of a
> single kafka cluster. This allows each micro-service in our enterprise to
> consume & process kafka data to suit its own needs.
> Currently when joining streams, an internal topic is created & named using
> KStreamsBuilder.newName(prefix);
> ```
> String thisWindowStreamName = topology.newName(WINDOWED_NAME);
> String otherWindowStreamName = topology.newName(WINDOWED_NAME);
> String joinThisName = rightOuter ?
> topology.newName(OUTERTHIS_NAME) : topology.newName(JOINTHIS_NAME);
> String joinOtherName = leftOuter ?
> topology.newName(OUTEROTHER_NAME) : topology.newName(JOINOTHER_NAME);
> String joinMergeName = topology.newName(MERGE_NAME);
> ```
> This prefix is a constant, and internally an incrementor is used to generate
> a unique (per KStreamBuilder instance) topic name for the topic.
> In situations where multiple KStreamBuilders are in use (for example,
> multiple different Kafka Streams applications) we are seeing collisions in
> topic names.
> Perhaps the join() methods should take a topic prefix overload to allow
> developers to provide unique names for these topics. Similar to the way other
> stateful processors work when having to provide a store name.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
