[
https://issues.apache.org/jira/browse/KAFKA-5605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16090812#comment-16090812
]
Connie Chen commented on KAFKA-5605:
------------------------------------
[~mjsax] helped me fix the issue. It appears I was sharing `application.id`
across all my `KafkaStream` instances in my application. Each instance of
`KafkaStream` under the same `application.id` must be processing the same
topics. So the fix is to create a unique `application.id` per topic. Then, my
service application will have N application.id processing N different topics,
each with a different transformation, and each with its own `KStreamBuilder`
and `KafkaStream`
https://github.com/conniec/examples/commit/ede9051734f414ce3bae5d760789d3fc9f60e769
> Multiple KStreams being created on application startup are not processing
> -------------------------------------------------------------------------
>
> Key: KAFKA-5605
> URL: https://issues.apache.org/jira/browse/KAFKA-5605
> Project: Kafka
> Issue Type: Bug
> Reporter: Connie Chen
> Attachments: app-logs.txt, server-logs.txt
>
>
> This is an issue filed in response to this slack discussion
> https://confluentcommunity.slack.com/archives/C48AHTCUQ/p1500079616414627
> I have a http service on top of Akka that starts and stops KStreams whose
> source/sink topics are configurable. Every time a new KStream is created I
> post a record to another Kafka topic `StreamConfig2`. On my application
> startup, I read from that config topic into a KTable and iterate records,
> create one KStream per record. However, it appears that this startup creation
> is not deterministic (ie. sometimes the KStreams get created properly,
> sometimes not). Also, not all KStreams are created.
> Matthias helpfully suggested that I change `application.id` every time on my
> application startup as a workaround. This seems to work for my application
> (service startup always creates the existing KStream in config topic),
> however I can't figure out why it doesn't work when I reuse `application.id`
> I have looked at https://issues.apache.org/jira/browse/KAFKA-5562 and tried
> increasing the config there, to no avail.
> Here I've attached logs from my application, expected logs, and kafka server
> logs.
> https://gist.github.com/conniec/baf0c011be8f29a4e09af0ceb136e33e
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
