Re: Flink source error handling

2023-06-12 Thread Martijn Visser
1. Like a regular Kafka client, so it depends on how you configure it. 2. Yes 3. It depends on the failure of course, but you could create something like a Dead Letter Queue in case deserialization fails for your incoming message. Best regards, Martijn On Sat, Jun 10, 2023 at 2:03 PM Anirban Dut

Re: Flink source error handling

2023-06-10 Thread Anirban Dutta Gupta
Hello, Thanks for the guidance. We will surely think of moving to a newer version of Flink. Just a few followup questions when using KafkaSource..(sorry if I am being naive in my questions) 1. How does KafkaSource handle disruptions with the Kafka broker ? Will it keep on trying to connect a

Re: Flink source error handling

2023-06-09 Thread Martijn Visser
Hi, This consumer should not be used. This only occurs in really old and no longer supported Flink versions. You should really upgrade to a newer version of Flink and use the KafkaSource. Best regards, Martijn On Fri, Jun 9, 2023 at 11:05 AM Anirban Dutta Gupta < anir...@indicussoftware.com> wr

Flink source error handling

2023-06-09 Thread Anirban Dutta Gupta
Hello, We are using "FlinkKafkaConsumer011" as a Kafka source consumer for Flink. Please guide on how to implement error handling mechanism for the following: 1. If the subscription to the Kafka topic gets lost, Kafka connection gets disconnected. In this case, is there any mechanism of re-sub