Re: Apache Kafka Streams : Out-of-Order messages & uses of TimeStamp extractor

2021-04-20 Thread Matthias J. Sax
Replied on StackOverflow: https://stackoverflow.com/questions/67158317/apache-kafka-streams-out-of-order-messages -Matthias On 4/20/21 4:21 PM, Neeraj Vaidya wrote: > Hi, > I have asked this on StackOverflow, but will ask it here as well. > > I have an Apache Kafka 2.6 Producer which writes

Apache Kafka Streams : Out-of-Order messages & uses of TimeStamp extractor

2021-04-20 Thread Neeraj Vaidya
Hi, I have asked this on StackOverflow, but will ask it here as well. I have an Apache Kafka 2.6 Producer which writes to topic-A (TA). I also have a Kafka streams application which consumes from TA and writes to topic-B (TB). In the streams application, I have a custom timestamp extractor

Fwd: Kafka Streams Commits

2021-04-20 Thread Geetish Nayak
Hello, I have a query with respect to *commits* in Kafka Streams. From what I understand,* commits to the broker are done by default every 30 seconds* by Kafka Streams and hence offset gets increased as a result. We have been seeing that these are done only for a short period of time(only during

Re: Kafka Stream: State replication seems unpredictable.

2021-04-20 Thread mangat rai
Hey Guozhang, Thanks for creating the issue. Yes, you are right, this will happen only with the consecutive rebalancing as after some time zombie thread will stop and re-join the group and the new thread will always overwrite the state with the latest data. In our poor infra setup, the

Fwd: Kafka Streams Commits

2021-04-20 Thread Geetish Nayak
Hello, I have a query with respect to *commits* in Kafka Streams. From what I understand,* commits to the broker are done by default every 30 seconds* by Kafka Streams and hence offset gets increased as a result. We have been seeing that these are done only for a short period of time(only during