Thanks a lot for working on this, long waiting and very demanded user feature.
I’ll try to take a look on design doc in the next days. — Alexey > On 8 Nov 2023, at 21:43, John Casey via dev <dev@beam.apache.org> wrote: > > Hi All, > > I've written up a design for adding DLQs to existing Beam IOs. It's been > through a round of reviews with some Dataflow folks at Google, but I'd > appreciate any comments the rest of Beam have around how to refine the design. > > TL;DR: Make it easy for a user to configure IOs to route bad data to an > alternate sink instead of crashing the pipeline or having the record be > retried indefinitely. > > https://docs.google.com/document/d/1NGeCk6tOqF-TiGEAV7ixd_vhIiWz9sHPlCa1P_77Ajs/edit?usp=sharing > > Thanks! > > John
