Re: Metric to capture decoding failure in flink sources

2023-10-11 Thread Prateek Kohli
Thanks a lot for your response Mason. Is there any FLIP planned to expose context in Reader in future? Regards, Prateek Kohli On Wed, Oct 11, 2023 at 6:03 AM Mason Chen wrote: > Hi Prateek, > > I agree, the reader should ideally expose the context to record metrics > about deserialization. One

Re: Metric to capture decoding failure in flink sources

2023-10-10 Thread Mason Chen
Hi Prateek, I agree, the reader should ideally expose the context to record metrics about deserialization. One option is to defer deserialization to another operator, say a RichMapFunction that has access to the RuntimeContext. Best, Mason On Mon, Oct 9, 2023 at 12:42 PM Prateek Kohli wrote: >

Metric to capture decoding failure in flink sources

2023-10-09 Thread Prateek Kohli
Hi, I need to get the difference between records which are collected by the source and the records which are emitted. For eg - If deserialization fails while reading records from kafka, in that case I want to expose the difference between records collected from Kafka Broker and records emitted fro