Re: Flink Iterations vs. While loop

Hi Dan, first a general remark: I fear that your L-BFGS implementation is not well suited for large scale problems. You might wanna take a look at [1]. In the case of the while loop solution you're actually executing n jobs with n being the number of iterations. Thus, you have to add the

Re: Sharing Java Collections within Flink Cluster

Hi Team, Can someone help me here? Appreciate any response ! Best Regards Varaga On Mon, Sep 5, 2016 at 4:51 PM, Chakravarthy varaga < chakravarth...@gmail.com> wrote: > Hi Team, > > I'm working on a Flink Streaming application. The data is injected > through Kafka connectors. The

Re: Sharing Java Collections within Flink Cluster

Hi, Flink does not provide shared state. However, you can broadcast a stream to CoFlatMapFunction, such that each operator has its own local copy of the state. If that does not work for you because the state is too large and if it is possible to partition the state (and both streams), you can

Re: Exception in CEP 1.1.2

Hi, I will try to get some minimal input to reproduce this. We were reading events from Kafka so I might need some time. Thanks Till for looking into this Gyula Till Rohrmann ezt írta (időpont: 2016. szept. 7., Sze, 17:34): > Hi Gyula, > > could you send us en example

Re: Sharing Java Collections within Flink Cluster

That depends. 1) Growing/Shrinking: This should work. New entries can always be inserted. In order to remove entries from the k-v-state you have to set the value to null. Note that you need an explicit delete-value record to trigger the eviction. 2) Multiple lookups: This does only work if all

Re: Sharing Java Collections within Flink Cluster

Is writing DataStream2 to a Kafka topic and reading it from the other job an option? 2016-09-07 19:07 GMT+02:00 Chakravarthy varaga : > Hi Fabian, > > Thanks for your response. Apparently these DataStream > (Job1-DataStream1 & Job2-DataStream2) are from different

Re: Sharing Java Collections within Flink Cluster

Operator state is always local in Flink. However, with key-value state, you can have something which behaves kind of similar to a distribute hashmap, because each operator holds a different shard/partition of the hashtable. If you have to do only a single key lookup for each element of DS1, you

Re: fsbackend with nfs

Hi CPC, It should be possible to use the FsBackend with NFS. However, I'm not sure how well it will perform. Regards, Robert On Mon, Sep 5, 2016 at 2:11 PM, CPC wrote: > Hi, > > Is it possible to use flinkstatebackend with nfs? We dont want to deploy > hadoop in our

Re: Sharing Java Collections within Flink Cluster

certainly, what I thought as well... The output of DataStream2 could be in 1000s and there are state updates... reading this topic from the other job, job1, is okie. However, assuming that we maintain this state into a collection, and updating the state (by reading from the topic) in this

Re: Exception in CEP 1.1.2

Interestingly on my local machine I could not reproduce the problem, maybe it was some build issue on the other machine. Have to investigate tomorrow :) Gyula Gyula Fóra ezt írta (időpont: 2016. szept. 7., Sze, 17:37): > Hi, > > I will try to get some minimal input to