Kafka Stream invalid partitions

2017-06-27 Thread D Stephan
Hello, When I use KafkaStreams DSL GroupByKey and Aggregate APIs, I have randomly & frequently below exceptions: In my opinion, it is not practical to clean up the invalid partitions everydays. For your information, this partition is an internal partition that automatically created by KafkaStream

Re: Kafka Stream invalid partitions

2017-06-27 Thread D Stephan
afka are you using? Also, between > runs do you change the number of partitions for your topics? I’m trying to > figure out how this problem happens, any information on what is changing in > between runs is appreciated. > > Thanks, > Eno > > > On Jun 27, 2017, at 8:52 AM, D St

Re: Kafka Stream invalid partitions

2017-06-27 Thread D Stephan
Thanks Eno, for the info!. I will try your suggestion. 2017-06-27 14:04 GMT+02:00 Eno Thereska : > Thanks. I believe we’ve addressed this issue in 0.10.2.1, any chance you > could try that? > > Thanks > Eno > > On Jun 27, 2017, at 11:14 AM, D Stephan wrote: > > &g

Kafka Stream Exception: partition issue

2017-11-20 Thread D Stephan
Hello, We are using Kafka version 0.11.0.1. with KafkaStream. Using the leftJoin API in the Kafka Stream: KStream leftJoin(KStream otherStream, ValueJoiner joiner, JoinWindows windows) https://kafka.apache.org/0110/javadoc/org/apache/kaf

Re: Kafka Stream Exception: partition issue

2017-11-20 Thread D Stephan
rokers. > > https://docs.confluent.io/current/streams/developer- > guide/config-streams.html#kafka-consumers-and-producer- > configuration-parameters > > Please let us know if this resolved the issue. > > > -Matthias > > On 11/20/17 7:22 AM, D Stephan wrote: > >

Kafka topic inconsistency

2017-12-01 Thread D Stephan
Hello, We have the following code to send avro message to a topic (Please find the code snippet below). Although the ProducerResord received a complete acknowledgement withiot exception, the message is never stored in this topic. In the log we can see the following message is logged: "collectedRe