Problem with CEPPatternOperator when taskmanager is killed

2016-09-14 Thread jaxbihani
*Problem :* I have created a PatternStream with "custom type" and added an event pattern. This works fine in both local and cluster setup. But when I tried to take one of the taskmanager down (on which task was executing), flink tries to restart a job but restart fails with the exception : "Could

Re: SQL for Flink

2016-09-14 Thread Greg Hogan
Hi Deepak, There are many open tickets for Flink's SQL API. Documentation is at https://ci.apache.org/projects/flink/flink-docs-master/dev/table_api.html.

Re: SQL for Flink

2016-09-14 Thread Deepak Sharma
Thanks Greg . I will start picking some of them. Thanks Deepak On 14 Sep 2016 6:31 pm, "Greg Hogan" wrote: > Hi Deepak, > > There are many open tickets for Flink's SQL API. Documentation is at > https://ci.apache.org/projects/flink/flink-docs-master/dev/table_api.html. > >

Re: Streaming issue help needed

2016-09-14 Thread Márton Balassi
Dear Vaidya, This seems weird, me guess is that somehow that Time and AbstractTime implementations are not from the same Flink version. According to your Maven build you should be using Flink 0.10.2. Since then there have been changes to windowing, are you tied to that version or would it be

CEP two transitions to the same state

2016-09-14 Thread Frank Dekervel
Hello, I'm trying to model a FSM using the flink CEP patterns. However, there is something i can't figure out as all the documentation examples are linear (either you go to the single possible next state, either no match). Suppose that two transitions lead from one state to two different states.

Re: Fw: Flink Cluster Load Distribution Question

2016-09-14 Thread amir bahmanyari
Hi Aljoscha,Thanks for your response. Its the same job but I am reading through TextIO() instead of a Kafka topic.I thought that would make a difference. It doesnt. Same slowness in Flink Cluster.I had sent you the code with reading from KafkaIO().Nothing different except commenting out the

Re: Fw: Flink Cluster Load Distribution Question

2016-09-14 Thread amir bahmanyari
Hi Aljoscha,The JM logs is also attached. Seems like everything is ok, assigned...to all nodes...Not sure why I dont get performance? :-(Thanks+regards,Amir- From: Aljoscha Krettek To: user@flink.apache.org; amir bahmanyari Sent: Wednesday,

SQL for Flink

2016-09-14 Thread Radu Tudoran
Hi, As a follow up to multiple discussions that happened during Flink Forward about how SQL should be supported by Flink, I was thinking to make a couple of proposals. Disclaimer: I do not claim I have managed to synthesized all the discussions and probably a great deal of things are still

Re: SQL for Flink

2016-09-14 Thread Deepak Sharma
+1 Yes.I agree to having SQL for Flink. I can take up some tasks as well once this starts. Thanks Deepak On Wed, Sep 14, 2016 at 3:47 PM, Radu Tudoran wrote: > Hi, > > > > As a follow up to multiple discussions that happened during Flink Forward > about how SQL should

Re: Data Transfer between TM should be encrypted

2016-09-14 Thread vinay patil
Hi Vijay, Did you raise the PR for this task, I don't mind testing it out as well. Regards, Vinay Patil On Tue, Aug 30, 2016 at 6:28 PM, Vinay Patil wrote: > Hi Vijay, > > That's a good news for me. Eagerly waiting for this change so that I can > integrate and test it

Re: Why tuples are not ignored after watermark?

2016-09-14 Thread Aljoscha Krettek
Hi, the problem might be that your timestamp/watermark assigner is run in parallel and that only one parallel instance of those operators emits the watermark because only one of those parallel instances sees the element with _3 == 9000. For the watermark to advance at an operator it needs to

Re: Tumbling window rich functionality

2016-09-14 Thread Aljoscha Krettek
Hi, WindowFunction.apply() will be called once for each window so you should be able to do the setup/teardown in there. open() and close() are called at the start of processing, end of processing, respectively. Cheers, Aljoscha On Wed, 14 Sep 2016 at 09:04 Swapnil Chougule