Re: Backpressure handling in FileSource APIs - Flink 1.16

2023-05-28 Thread Shammon FY
monitoring/back_pressure/ >>>> >>>> On Tue, May 23, 2023 at 9:40 PM Kamal Mittal >>>> wrote: >>>> >>>>> Hello Community, >>>>> >>>>> Can you please share views about the query asked above w.r.t back >>

Re: Backpressure handling in FileSource APIs - Flink 1.16

2023-05-26 Thread Kamal Mittal
d above w.r.t back >>>> pressure for FileSource APIs for Bulk and Record stream formats. >>>> Planning to use these APIs w.r.t AVRO to Parquet and vice-versa >>>> conversion. >>>> >>>> Rgds, >>>> Kamal >>>> >>>> On Tu

Re: Backpressure handling in FileSource APIs - Flink 1.16

2023-05-24 Thread Kamal Mittal
gt;> Rgds, >>> Kamal >>> >>> On Tue, 23 May 2023, 12:26 pm Kamal Mittal, wrote: >>> >>>> Added Flink community DL as well. >>>> >>>> -- Forwarded message - >>>> From: Kamal Mittal >>&g

Re: Backpressure handling in FileSource APIs - Flink 1.16

2023-05-24 Thread Kamal Mittal
>> >>> -- Forwarded message - >>> From: Kamal Mittal >>> Date: Tue, May 23, 2023 at 7:57 AM >>> Subject: Re: Backpressure handling in FileSource APIs - Flink 1.16 >>> To: Shammon FY >>> >>> >>> Hello, >

Re: Backpressure handling in FileSource APIs - Flink 1.16

2023-05-23 Thread Shammon FY
te: > >> Added Flink community DL as well. >> >> -- Forwarded message - >> From: Kamal Mittal >> Date: Tue, May 23, 2023 at 7:57 AM >> Subject: Re: Backpressure handling in FileSource APIs - Flink 1.16 >> To: Shammon FY >> >>

Re: Backpressure handling in FileSource APIs - Flink 1.16

2023-05-23 Thread Kamal Mittal
: > Added Flink community DL as well. > > -- Forwarded message - > From: Kamal Mittal > Date: Tue, May 23, 2023 at 7:57 AM > Subject: Re: Backpressure handling in FileSource APIs - Flink 1.16 > To: Shammon FY > > > Hello, > > Yes, want to take some cus

Fwd: Backpressure handling in FileSource APIs - Flink 1.16

2023-05-23 Thread Kamal Mittal
Added Flink community DL as well. -- Forwarded message - From: Kamal Mittal Date: Tue, May 23, 2023 at 7:57 AM Subject: Re: Backpressure handling in FileSource APIs - Flink 1.16 To: Shammon FY Hello, Yes, want to take some custom actions and also if there is any default

Re: Backpressure handling in FileSource APIs - Flink 1.16

2023-05-22 Thread Shammon FY
Hi Kamal, If I understand correctly, do you want the source to do some custom actions, such as current limiting, when there is backpressure in the job? Best, Shammon FY On Mon, May 22, 2023 at 2:12 PM Kamal Mittal wrote: > Hello Community, > > Can you please share views about the query asked

Re: Backpressure handling in FileSource APIs - Flink 1.16

2023-05-22 Thread Kamal Mittal
Hello Community, Can you please share views about the query asked above w.r.t back pressure for FileSource APIs for Bulk and Record stream formats. Planning to use these APIs w.r.t AVRO to Parquet and vice-versa conversion. Rgds, Kamal On Thu, May 18, 2023 at 2:33 PM Kamal Mittal wrote: >

Backpressure handling in FileSource APIs - Flink 1.16

2023-05-18 Thread Kamal Mittal
Hello Community, Does FileSource APIs for Bulk and Record stream formats handle back pressure by any way like slowing down sending data in piepline further or reading data from source somehow? Or does it give any callback/handle so that any action can be taken? Can you please share details if