Re: [DISCUSS] FLIP-451: Refactor Async sink API

2024-05-22 Thread Leonard Xu
he/flink/blob/015867803ff0c128b1c67064c41f37ca0731ed86/flink-connectors/flink-connector-base/src/main/java/org/apache/flink/connector/base/sink/throwable/FatalExceptionClassifier.java#L32 >>>>>>> 2- >>>>>>> >>>>>>> >>>>>> &

Re: [DISCUSS] FLIP-451: Refactor Async sink API

2024-05-22 Thread Ahmed Hamdy
link-connector-base/src/main/java/org/apache/flink/connector/base/sink/writer/AsyncSinkWriter.java#L533 > >>>>> 3- > >>>>> > >>>>> > >>>> > >> > https://github.com/apache/flink-connector-aws/blob/c6e0ab

Re: [DISCUSS] FLIP-451: Refactor Async sink API

2024-05-21 Thread Leonard Xu
gt;> 3- >>>>> >>>>> >>>> >> https://github.com/apache/flink-connector-aws/blob/c6e0abb65a0e51b40dd218b890a111886fbf797f/flink-connector-aws/flink-connector-aws-kinesis-streams/src/main/java/org/apache/flink/connector/kinesis/sink/KinesisStreamsSink

Re: [DISCUSS] FLIP-451: Refactor Async sink API

2024-05-21 Thread Ahmed Hamdy
/sink/KinesisStreamsSinkWriter.java#L106 > >>> > >>> Best Regards > >>> Ahmed Hamdy > >>> > >>> > >>> On Mon, 13 May 2024 at 16:20, David Radley > >>> wrote: > >>> > >>>> Hi, > >>>>

Re: [DISCUSS] FLIP-451: Refactor Async sink API

2024-05-21 Thread Leonard Xu
e (transient) >>>> errors (like IOExceptions) . I see some talk on the internet around >>>> retriable SQL errors. >>>> If this was the case then we may need configuration to limit the >> number >>>> of retries of retriable errors. >>>>

Re: [DISCUSS] FLIP-451: Refactor Async sink API

2024-05-21 Thread Ahmed Hamdy
t) > > > errors (like IOExceptions) . I see some talk on the internet around > > > retriable SQL errors. > > > If this was the case then we may need configuration to limit the > number > > > of retries of retriable errors. > > > Kind r

Re: [DISCUSS] FLIP-451: Refactor Async sink API

2024-05-20 Thread Hong Liang
gt; If this was the case then we may need configuration to limit the number > > of retries of retriable errors. > > Kind regards, David > > > > > > From: Muhammet Orazov > > Date: Monday, 13 May 2024 at 10:30 > > To: dev@flink.apache.org >

Re: [DISCUSS] FLIP-451: Refactor Async sink API

2024-05-13 Thread Ahmed Hamdy
umber > of retries of retriable errors. > Kind regards, David > > > From: Muhammet Orazov > Date: Monday, 13 May 2024 at 10:30 > To: dev@flink.apache.org > Subject: [EXTERNAL] Re: [DISCUSS] FLIP-451: Refactor Async sink API > Great, thanks for clarifying! > > Best

RE: [DISCUSS] FLIP-451: Refactor Async sink API

2024-05-13 Thread David Radley
to limit the number of retries of retriable errors. Kind regards, David From: Muhammet Orazov Date: Monday, 13 May 2024 at 10:30 To: dev@flink.apache.org Subject: [EXTERNAL] Re: [DISCUSS] FLIP-451: Refactor Async sink API Great, thanks for clarifying! Best, Muhammet On 2024-05-06 13

Re: [DISCUSS] FLIP-451: Refactor Async sink API

2024-05-13 Thread Muhammet Orazov
Great, thanks for clarifying! Best, Muhammet On 2024-05-06 13:40, Ahmed Hamdy wrote: Hi Muhammet, Thanks for the feedback. Could you please add more here why it is harder? Would the `completeExceptionally` method be related to it? Maybe you can add usage example for it also. this is

Re: [DISCUSS] FLIP-451: Refactor Async sink API

2024-05-06 Thread Ahmed Hamdy
Hi Muhammet, Thanks for the feedback. > Could you please add more here why it is harder? Would the > `completeExceptionally` > method be related to it? Maybe you can add usage example for it also. > this is mainly due to the current implementation of fatal exception failures which depends on

Re: [DISCUSS] FLIP-451: Refactor Async sink API

2024-05-06 Thread Muhammet Orazov
Hey Ahmed, Thanks for the FLIP! +1 (non-binding) Additionally the current interface for passing fatal exceptions and retrying records relies on java consumers which makes it harder to understand. Could you please add more here why it is harder? Would the `completeExceptionally` method be

Re: [DISCUSS] FLIP-451: Refactor Async sink API

2024-05-04 Thread Ahmed Hamdy
Hi Jeyhun, I have changed the scope of FLIP to exclude problems addressed by FLIP-284 and redefined scope to introduce timeout configuration. Best Regards Ahmed Hamdy On Mon, 29 Apr 2024 at 20:57, Ahmed Hamdy wrote: > Hi Jeyhun, > Thanks for your feedback. I agree the phrasing is a bit

Re: [DISCUSS] FLIP-451: Refactor Async sink API

2024-04-29 Thread Ahmed Hamdy
Hi Jeyhun, Thanks for your feedback. I agree the phrasing is a bit confusing, the main scope for FLIP-451 is limited to introducing timeout configuration and the new "ResultHandler" to Async Sink API. I will remove reference to FLIP-284 from the FLIP to disambiguate. Best Regards Ahmed Hamdy On

Re: [DISCUSS] FLIP-451: Refactor Async sink API

2024-04-29 Thread Jeyhun Karimov
Hi Ahmed, Thanks a lot for the FLIP. +1 for it. My main concern is that the boundary/scope of the two FLIPs (451 and 284) and their differentiation/overlap is unclear for me from the FLIP document. Could you please elaborate more on this? Regards, Jeyhun On Mon, Apr 29, 2024 at 4:13 PM Ahmed