Hi, Flink does not provide shared state. However, you can broadcast a stream to CoFlatMapFunction, such that each operator has its own local copy of the state.
If that does not work for you because the state is too large and if it is possible to partition the state (and both streams), you can also use keyBy instead of broadcast. Finally, you can use an external system like a KeyValue Store or In-Memory store like Apache Ignite to hold your distributed collection. Best, Fabian 2016-09-07 17:49 GMT+02:00 Chakravarthy varaga <chakravarth...@gmail.com>: > Hi Team, > > Can someone help me here? Appreciate any response ! > > Best Regards > Varaga > > On Mon, Sep 5, 2016 at 4:51 PM, Chakravarthy varaga < > chakravarth...@gmail.com> wrote: > >> Hi Team, >> >> I'm working on a Flink Streaming application. The data is injected >> through Kafka connectors. The payload volume is roughly 100K/sec. The event >> payload is a string. Let's call this as DataStream1. >> This application also uses another DataStream, call it DataStream2, >> (consumes events off a kafka topic). The elements of this DataStream2 >> involves in a certain transformation that finally updates a Hashmap(/Java >> util Collection). Apparently the flink application should share this >> HashMap across the flink cluster so that DataStream1 application could >> check the state of the values in this collection. Is there a way to do this >> in Flink? >> >> I don't see any Shared Collection used within the cluster? >> >> Best Regards >> CVP >> > >
