[Question] - How can i listen to multiple Pub/Sub input topics using SideInput?

2023-10-31 Thread Pravin DSouza
Hi, I have a use case where I want to listen to multiple Pub/Sub input topics and route messages from all multiple input topics to a single destination output topic. The number of input topics can change any time and are stored in a table. For example when I deploy for the first time, I might be

Re: [Question] - How can i listen to multiple Pub/Sub input topics using SideInput?

2023-11-07 Thread Pravin DSouza
gt; from Cloud Pub/Sub to support this. Also, probably you'll have to constrain > reading with size/time since Cloud Pub/Sub is an unbounded input source. > > Thanks, > Cham > > > > On Tue, Oct 31, 2023 at 2:22 PM Pravin DSouza wrote: > >> Hi, >> >> I h

Re: [Question] - How can i listen to multiple Pub/Sub input topics using SideInput?

2023-11-07 Thread Pravin DSouza
n >> reading with size/time since Cloud Pub/Sub is an unbounded input source. >> >> Thanks, >> Cham >> >> >> >> On Tue, Oct 31, 2023 at 2:22 PM Pravin DSouza >> wrote: >> >>> Hi, >>> >>> I have a use case where I w

Re: [Question] - How can i listen to multiple Pub/Sub input topics using SideInput?

2023-11-07 Thread Pravin DSouza
/docs/guides/updating-a-pipeline#Launching > > > Thanks, > Cham > > On Tue, Nov 7, 2023 at 9:25 AM Pravin DSouza wrote: > >> Hi Wisniowski, >> >> Thank you for the inputs. >> It is not possible for us to identify all the topics in advance as new >> r