Hi, For Ignite, Flink has a Sink, which is a one-directional thing. I think that Sebastian needs a bi-directional connection. An in-memory KV store like redis or memcache is probably the best option for such a use case (it reminds me a bit of the Yahoo streaming benchmark [1]).
[1] https://yahooeng.tumblr.com/post/135321837876/benchmarking-streaming-computation-engines-at On Sat, Jul 9, 2016 at 4:03 PM, Suneel Marthi <suneel.mar...@gmail.com> wrote: > U could use ignite too, I believe they have a plugin for flink streaming. > > Sent from my iPhone > > > On Jul 9, 2016, at 8:05 AM, Sebastian <s...@apache.org> wrote: > > > > Hi, > > > > I'm planning to work on a streaming recommender in Flink, and one > problem that I have is that the algorithm needs random access to a small > global state (say a million counts). It should be ok if there is some > inconsistency in the state (e.g., delay in seeing updates). > > > > Does anyone here have experience with such things? I'm thinking of > connecting Flink to a lighweight in-memory key-value store such as memcache > for that. > > > > Best, > > Sebastian >
