Hi! Is it possible adding the topic name in the pubsub message to the internal DataFlow implementation?
Thanks, Tamir בתאריך יום ג׳, 28 בדצמ׳ 2021 ב-22:19 מאת Ahmet Altay <[email protected] >: > Adding people who might be able to comment on : @Reuven Lax > <[email protected]> @Dustin Rhodes <[email protected]> @Slava Chernyak > <[email protected]> > > Most of these people are out of office, and they probably would not > respond this week. > > On Tue, Dec 28, 2021 at 1:29 AM tamir yardeni <[email protected]> > wrote: > >> Hi everyone, >> >> I’m working on a PR to PubsubIO that will allow writing to different >> topics dynamically [1]. >> >> PubsubUnboundedSink is replaced by DataFlow runner with internal >> implementation so it requires changes in the runner. >> >> I was able to run the changes in DataFlow by adding the flag >> “enable_custom_pubsub_sink“ but I guess there is a good reason for the >> internal replacement. >> >> Can someone help with those changes? >> >> >> It was inspired by "BigQuery dynamic destination" and this will have a >> huge value for my team. >> >> >> [1] - https://github.com/apache/beam/pull/16366 >> >> >> Thanks! >> >> Tamir >> >
