Re: Kafka Streams: Unexpected data loss

2020-11-22 Thread Jeffrey Goderie
Hi John, Thank you for your swift response and the advice. Your answers confirm our expectations. As you suggested we'll keep monitoring the application to see whether a similar issue occurs. For now every analysis points towards the behaviour being an effect of circumstance rather than a

Re: Kafka Streams: Unexpected data loss

2020-11-20 Thread John Roesler
Hello Jeffrey, I’m sorry for the trouble. I appreciate your diligence in tracking this down. In reading your description, nothing jumps out to me as problematic. I’m a bit at a loss as to what may have been the problem. >- Is there a realistic scenario (e.g. crash, rebalance) which you

Kafka Streams: Unexpected data loss

2020-11-20 Thread Jeffrey Goderie
Hi all, We recently started using Kafka Streams and we encountered an unexpected issue with our Streams application. Using the following topology we ran into data loss: Topologies: Sub-topology: 0 Source: meteringpoints-source (topics: [serving.meteringpoints.mad-meteringpoint]) -->

Kafka Streams: Unexpected data loss

2020-11-20 Thread Jeffrey Goderie
Hi all, We recently started using Kafka Streams and we encountered an unexpected issue with our Streams application. Using the following topology we ran into data loss: Topologies: Sub-topology: 0 Source: meteringpoints-source (topics: [serving.meteringpoints.mad-meteringpoint]) -->