reuvenlax commented on issue #21713: URL: https://github.com/apache/beam/issues/21713#issuecomment-1251231519
In general, that pattern is avoided in streaming, since the system prioritizes data consistency. Temporary problems (e.g. temporary issues with the sink, the network, or anything else) could easily cause N retries to exceed, causing 100% of output going to the failure collection. A second issue is that there isn't a good way of determining retry count. We can only know retry count in the current bundle. If the bundle fails and retries, our count gets set back to 0. On Mon, Sep 19, 2022 at 8:36 AM johnjcasey ***@***.***> wrote: > Is there a particular reason to not use a pattern of outputting to failure > after N retries? > > — > Reply to this email directly, view it on GitHub > <https://github.com/apache/beam/issues/21713#issuecomment-1251190989>, or > unsubscribe > <https://github.com/notifications/unsubscribe-auth/AFAYJVMQ6RMY6N4A66IOEZTV7CCA3ANCNFSM5X4IOSHA> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> > -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
