Hi! A global shared variable is not something that is offered by Flink right now. It is not part of the system, because it is not really part of the stream or state derived from individual streams. It is also quite hard to do efficiently and general purpose.
I see that it is a useful tool in several use cases. You would currently have to add that yourself, you can try and use something like Redis to hold shared state, or you can use an embedded Akka and CRDTs to have replicated state that syncs up across all nodes. Greetings, Stephan On Wed, Aug 10, 2016 at 6:18 PM, Dima Arbuzin <arbuzins...@gmail.com> wrote: > Hi there, > > I'm working on a clustering algorithm on a DataStream based on this paper > <http://www1.cse.wustl.edu/~ychen/public/sigproc-sp.pdf> and in order to > implement that, I need to maintain some global values (initially HashMap > but might be DataSet) and update it on every item arriving. > > If this is even possible with Flink? (some kind of reverse broadcasting) > And if yes, what would be the right way to do it? or what are the > alternatives? > > As a stream simulation I use source function based on the demo > <https://github.com/dataArtisans/flink-streaming-demo/blob/master/src/main/scala/com/dataartisans/flink_demo/sources/TaxiRideSource.scala> > . > > Thanks, > > > -- > Best regards, > Dima Arbuzin >
