Re: How can I prove ....

2016-09-23 Thread Stephan Ewen
Are you sure you have the parallelism set to 448? You can see the parallelism of operators in the web UI. On Fri, Sep 23, 2016 at 12:15 AM, amir bahmanyari wrote: > Hi Again & sorry to take your time. But am puzzled by what I cannot > explain why. > The parallelism is set

Re: Simple batch job hangs if run twice

2016-09-23 Thread Yassine MARZOUGUI
Hi Fabian, Not sure if this answers your question, here is the stack I got when debugging the combine and datasource operators when the job got stuck: "DataSource (at main(BatchTest.java:28) (org.apache.flink.api.java.io.TupleCsvInputFormat)) (1/8)" at java.lang.Object.wait(Object.java) at

Flink Checkpoint runs slow for low load stream

2016-09-23 Thread Chakravarthy varaga
Hi Aljoscha & Fabian, I have a stream application that has 2 stream source as below. KeyedStream *ks1* = ds1.keyBy("*") ; KeyedStream, String> *ks2* = ds2.flatMap(split T into k-v pairs).keyBy(0); ks1.connect(ks2).flatMap(X); //X is a

Re: Simple batch job hangs if run twice

2016-09-23 Thread Fabian Hueske
Hi Yassine, can you share a stacktrace of the job when it got stuck? Thanks, Fabian 2016-09-22 14:03 GMT+02:00 Yassine MARZOUGUI : > The input splits are correctly assgined. I noticed that whenever the job > is stuck, that is because the task *Combine (GroupReduce at

window-like use case

2016-09-23 Thread Maciek Próchniak
Hi, in our project we're dealing with a stream of billing events. Each has customerId and charge amount We want to have a process that will trigger event (alarm) when sum of charges for customer during last 4 hours exceeds certain threshold, say - 10. The triggered event should contain data

Re: Rich Window Function - When does close(tear down) method executes ?

2016-09-23 Thread Swapnil Chougule
Can I get any update please ? Regards, Swapnil