Hi Gopi, What about using a window with a custom trigger? The window is doing nothing but aggregating your input to a collection. The trigger accepts metadata from the low input stream so it can fire and purge the window (emit all elements in the window to downstream) on arrival of metadata.
Best, Qingsheng > On Jun 27, 2022, at 12:46, Gopi Krishna M <gopikrish...@gmail.com> wrote: > > Hi, > I've a scenario where I use connected streams where one is a low throughput > metadata stream and another one is a high throughput data stream. I use > CoProcessFunction that operates on a data stream with behavior controlled by > a metadata stream. > > Is there a way to slow down/pause the high throughput data stream until I've > received one entry from the metadata stream? It's possible that by the time I > get the first element from the metadata stream, I might get 1000s of items > from the data stream. One option is to create a state to buffer the data > stream within the operator. Is there any other option which doesn't need this > state management? > > Thanks, > Gopi
