Hi all, I have a question regarding error being thrown in KafkaWriter.processElement(). Let's say my pipeline eventually reached KafkaWriter.processElement() twice. The first time is successful and the second time, for some reason is not successful and set numSendFailures to 1.
After that DoFnOperator.checkInvokeFinishBundleByTime kicks in and invokes KafkaWriter.finishBundle() and it throws an IOException because numSendFailures is not 0. Does it mean that I will have some data lost and my application will need to handle it somehow? What would be the a recommended way of error handling in Beam pipelines? Thanks in advance. Antonio.
