Hi Kamal, If I understand correctly, do you want the source to do some custom actions, such as current limiting, when there is backpressure in the job?
Best, Shammon FY On Mon, May 22, 2023 at 2:12 PM Kamal Mittal <kspmm...@gmail.com> wrote: > Hello Community, > > Can you please share views about the query asked above w.r.t back pressure > for FileSource APIs for Bulk and Record stream formats. > Planning to use these APIs w.r.t AVRO to Parquet and vice-versa conversion. > > Rgds, > Kamal > > On Thu, May 18, 2023 at 2:33 PM Kamal Mittal <kspmm...@gmail.com> wrote: > >> Hello Community, >> >> Does FileSource APIs for Bulk and Record stream formats handle back >> pressure by any way like slowing down sending data in piepline further or >> reading data from source somehow? >> Or does it give any callback/handle so that any action can be taken? Can >> you please share details if any? >> >> Rgds, >> Kamal >> >
