Re: Broadcasting sets in Flink Streaming

2015-08-25 Thread Tamara Mendt
Ok, I'll try that. Thanks a lot! On Tue, Aug 25, 2015 at 4:19 PM, Stephan Ewen se...@apache.org wrote: You can do something very similar like broadcast sets like this: Use a Co-Map function and connect your main data set regularly (forward partitioning) to one input and your broadcast set

Re: Broadcasting sets in Flink Streaming

2015-08-25 Thread Stephan Ewen
You can do something very similar like broadcast sets like this: Use a Co-Map function and connect your main data set regularly (forward partitioning) to one input and your broadcast set via broadcast to the other input. You can then retrieve the data in the two map functions separately. This

Broadcasting sets in Flink Streaming

2015-08-25 Thread Tamara Mendt
Hello, I have been trying to use the function withBroadcastSet on a SingleOutputStreamOperator (map) the same way I would on a MapOperator for a DataSet. From what I see, this cannot be done. I wonder if there is some way to broadcast a DataSet to the tasks that are performing transformations on

Re: Broadcasting sets in Flink Streaming

2015-08-25 Thread Till Rohrmann
Hi Tamara, I think this is not officially supported by Flink yet. However, I think that Gyula had once an example where he did something comparable. Maybe he can chime in here. Cheers, Till On Tue, Aug 25, 2015 at 11:15 AM, Tamara Mendt tammyme...@gmail.com wrote: Hello, I have been trying