For the moment, yes. On Wed, Dec 6, 2023 at 11:21 AM Juan Romero <[email protected]> wrote:
> Thanks John. Is it the same case if i want to write in a postgres table > with the sql connector? > > El mié, 6 dic 2023 a las 11:05, John Casey (<[email protected]>) > escribió: > >> It is, but it's not possible to to take an existing transform, and simply >> configure it to do this. >> >> For example (and this is what I'm doing), it's possible to write a >> transform that tries to write to kafka, and upon failure, emits the failure >> to an alternate pcollection. >> >> It's not possible (yet) to take an existing PTransform that's part of the >> library, and configure it to do something other than simply retrying >> failures >> >> On Wed, Dec 6, 2023 at 10:44 AM Juan Romero <[email protected]> wrote: >> >>> But , is it not possible to get the message that can't reach the target >>> sink and put it in another target (eg: kafka error topic where we can >>> verify which messages failed to be delivered to the target)? >>> >>> >>> El mié, 6 dic 2023 a las 10:40, John Casey via user (< >>> [email protected]>) escribió: >>> >>>> I'm currently implementing improvements on Kafka, File, Spanner, and >>>> Bigtable IOs. >>>> >>>> I'm planning on tackling PubSub and BQ next year. >>>> >>>> All of this is still in progress though, so there aren't easy >>>> workarounds for the moment. >>>> >>>> On Tue, Dec 5, 2023 at 5:56 PM Robert Bradshaw <[email protected]> >>>> wrote: >>>> >>>>> Currently error handling is implemented on sinks in an ad-hoc basis >>>>> (if at all) but John (cc'd) is looking at improving things here. >>>>> >>>>> On Mon, Dec 4, 2023 at 10:25 AM Juan Romero <[email protected]> wrote: >>>>> > >>>>> > Hi guys. I want to ask you about how to deal with the scenario when >>>>> the target sink (eg: jdbc, kafka, bigquery, pubsub etc) fails for any >>>>> reason and i don't want to lost the message and create a bottleneck with >>>>> many errors due an hypothetical target sink problem, and i want to use >>>>> with_excpetion_handling in order to get the message that failing to reach >>>>> the target and send the message to an other error topic. Any idea to solve >>>>> this scenario? >>>>> >>>>
