[
https://issues.apache.org/jira/browse/KAFKA-5605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Connie Chen updated KAFKA-5605:
-------------------------------
Description:
This is an issue filed in response to this slack discussion
https://confluentcommunity.slack.com/archives/C48AHTCUQ/p1500079616414627
I have a http service on top of Akka that starts and stops KStreams whose
source/sink topics are configurable. Every time a new KStream is created I post
a record to another Kafka topic `StreamConfig2`. On my application startup, I
read from that config topic into a KTable and iterate records, create one
KStream per record. However, it appears that this startup creation is not
deterministic (ie. sometimes the KStreams get created properly, sometimes not).
Also, not all KStreams are created.
Matthias helpfully suggested that I change `application.id` every time on my
application startup as a workaround. This seems to work for my application
(service startup always creates the existing KStream in config topic), however
I can't figure out why it doesn't work when I reuse `application.id`
I have looked at https://issues.apache.org/jira/browse/KAFKA-5562 and tried
increasing the config there, to no avail.
Here I've attached logs from my application, expected logs, and kafka server
logs.
https://gist.github.com/conniec/baf0c011be8f29a4e09af0ceb136e33e
was:
This is an issue filed in response to this slack discussion
https://confluentcommunity.slack.com/archives/C48AHTCUQ/p1500079616414627
I have a http service on top of Akka that starts and stops KStreams whose
source/sink topics are configurable. Every time a new KStream is created I post
a record to another Kafka topic `StreamConfig2`. On my application startup, I
read from that config topic into a KTable and create existing KStreams.
However, it appears that this startup creation is not deterministic (ie.
sometimes the KStreams get created properly, sometimes not).
Matthias helpfully suggested that I change `application.id` every time on my
application startup as a workaround. This seems to work for my application
(service startup always creates the existing KStream in config topic), however
I can't figure out why it doesn't work when I reuse `application.id`
I have looked at https://issues.apache.org/jira/browse/KAFKA-5562 and tried
increasing the config there, to no avail.
Here I've attached logs from my application, expected logs, and kafka server
logs.
https://gist.github.com/conniec/baf0c011be8f29a4e09af0ceb136e33e
> Multiple KStreams being created on application startup are not processing
> -------------------------------------------------------------------------
>
> Key: KAFKA-5605
> URL: https://issues.apache.org/jira/browse/KAFKA-5605
> Project: Kafka
> Issue Type: Bug
> Reporter: Connie Chen
> Attachments: app-logs.txt, server-logs.txt
>
>
> This is an issue filed in response to this slack discussion
> https://confluentcommunity.slack.com/archives/C48AHTCUQ/p1500079616414627
> I have a http service on top of Akka that starts and stops KStreams whose
> source/sink topics are configurable. Every time a new KStream is created I
> post a record to another Kafka topic `StreamConfig2`. On my application
> startup, I read from that config topic into a KTable and iterate records,
> create one KStream per record. However, it appears that this startup creation
> is not deterministic (ie. sometimes the KStreams get created properly,
> sometimes not). Also, not all KStreams are created.
> Matthias helpfully suggested that I change `application.id` every time on my
> application startup as a workaround. This seems to work for my application
> (service startup always creates the existing KStream in config topic),
> however I can't figure out why it doesn't work when I reuse `application.id`
> I have looked at https://issues.apache.org/jira/browse/KAFKA-5562 and tried
> increasing the config there, to no avail.
> Here I've attached logs from my application, expected logs, and kafka server
> logs.
> https://gist.github.com/conniec/baf0c011be8f29a4e09af0ceb136e33e
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
