Thanks for the feedback @Till.
Yes I agree as well that opening up or changing the AsyncWaitOperator
doesn't seem to be a necessity here.
I think making "AsyncFunctionBase", making the current AsyncFunction as a
extension of it with a some of the default behaviors like Shuyi suggested
seems to be
Sorry for joining the discussion so late. I agree that we could add some
more syntactic sugar for handling failure cases. Looking at the existing
interfaces, I think it should be fairly easy to create an abstract class
AsyncFunctionWithRetry which extends AsyncFunction and encapsulates the
retry
Thanks for raising the concern @shuyi and the explanation @konstantin.
Upon glancing on the Flink document, it seems like user have full control
on the timeout behavior [1]. But unlike AsyncWaitOperator, it is not
straightforward to access the internal state of the operator to, for
example, put
Hi Shuyi,
I am not sure. You could handle retries in the user code within
org.apache.flink.streaming.api.functions.async.AsyncFunction#asyncInvoke
without using a DLQ as described in my original answer to William. On the
other hand, I agree that it could easier for the user and it is indeed a
Hi Konstantin,
(cc Till since he owns the code)
For async-IO, IO failure and retry is a common & expected pattern. In most
of the use cases, users will need to deal with IO failure and retry.
Therefore, I think it's better to address the problem in Flink rather than
user implementing its custom
Hi William,
the AsyncOperator does not have such a setting. It is "merely" a wrapper
around an asynchronous call, which provides integration with Flink's state
& time management.
I think, the way to go would be to do the exponential back-off in the user
code and set the timeout of the
Hi,
Is there a way to specify an exponential backoff strategy for when
async function calls fail?
I have an async function that does web requests to a rate-limited API.
Can you handle that with settings on the async function call?
Thanks,
William
