Mutable objects in Flink ?

2015-11-06 Thread Hajira Jabeen
Hi all, I am writing an evolutionary computing application with Flink. Each object is a particle with multidimensional velocity , position, and fitness. I create a population ( Array) of these objects and want to update positions and velocities of all particles in parallel. I can create the popul

Re: Apache Flink Operator State as Query Cache

2015-11-06 Thread Anwar Rizal
Let me understand your case better here. You have a stream of model and stream of data. To process the data, you will need a way to access your model from the subsequent stream operations (map, filter, flatmap, ..). I'm not sure in which case Operator State is a good choice, but I think you can als

Re: Published test artifacts for flink streaming

2015-11-06 Thread Till Rohrmann
No that is not possible since you cannot access DataSets from inside UDFs. And select and where operations are translated into a filter operation on a DataSet. ​ On Fri, Nov 6, 2015 at 6:03 PM, Nick Dimiduk wrote: > Promising observation, Till. Is it possible to access Table API's > select and w

Re: Published test artifacts for flink streaming

2015-11-06 Thread Nick Dimiduk
Promising observation, Till. Is it possible to access Table API's select and where operators from within such a flatMap? -n On Fri, Nov 6, 2015 at 6:19 AM, Till Rohrmann wrote: > Hi Nick, > > I think a flatMap operation which is instantiated with your list of > predicates should do the job. Thus

Re: Published test artifacts for flink streaming

2015-11-06 Thread Till Rohrmann
Hi Nick, I think a flatMap operation which is instantiated with your list of predicates should do the job. Thus, there shouldn’t be a need to dig deeper than the DataStream for the first version. Cheers, Till ​ On Fri, Nov 6, 2015 at 3:58 AM, Nick Dimiduk wrote: > Thanks Stephan, I'll check th

Apache Flink Operator State as Query Cache

2015-11-06 Thread Welly Tambunan
Hi All, We have a high density data that required a downsample. However this downsample model is very flexible based on the client device and user interaction. So it will be wasteful to precompute and store to db. So we want to use Apache Flink to do downsampling and cache the result for subseque