But , is it not possible to get the message that can't reach the target sink and put it in another target (eg: kafka error topic where we can verify which messages failed to be delivered to the target)?
El mié, 6 dic 2023 a las 10:40, John Casey via user (<[email protected]>) escribió: > I'm currently implementing improvements on Kafka, File, Spanner, and > Bigtable IOs. > > I'm planning on tackling PubSub and BQ next year. > > All of this is still in progress though, so there aren't easy workarounds > for the moment. > > On Tue, Dec 5, 2023 at 5:56 PM Robert Bradshaw <[email protected]> > wrote: > >> Currently error handling is implemented on sinks in an ad-hoc basis >> (if at all) but John (cc'd) is looking at improving things here. >> >> On Mon, Dec 4, 2023 at 10:25 AM Juan Romero <[email protected]> wrote: >> > >> > Hi guys. I want to ask you about how to deal with the scenario when the >> target sink (eg: jdbc, kafka, bigquery, pubsub etc) fails for any reason >> and i don't want to lost the message and create a bottleneck with many >> errors due an hypothetical target sink problem, and i want to use >> with_excpetion_handling in order to get the message that failing to reach >> the target and send the message to an other error topic. Any idea to solve >> this scenario? >> >
