Hej!
Yes, the "getRuntimeEnvironment().getBroadcastVariable() returns a list,
which you can add to set:
// in the function:
private Set<T> specials;
public void open(Configuration conf) {
List<T> bc =
getRuntimeContect().getBroadcastVariable("the-bc-var-name");
specials = new HashSet<T>(bc);
}
Is that what you had in mind?
Stephan
On Mon, Oct 6, 2014 at 11:57 AM, Martin Neumann <[email protected]>
wrote:
> Hej,
>
> I have a Flink job with with a filter step. I now have a list of exceptions
> where I need to do some extra work (300k data). I thought I just use a
> boradcast set and then for each like compare if its in the exception set.
>
> What is the best way to implement this in Flink? Is there an efficient way
> of checking if a certain element is in the Broadcastset? Or can I somehow
> dump the Broadcastset into a set?
>
> cheers Martin
>
