Job is be cancelled, but the stdout log still prints

2018-03-07 Thread sundy
Hi: I faced a problem, the taskmanagers in 3 nodes are still running, I make sure that all job are cancelled, but I could see that stdout logs are still printing all the way. The job's parallelism is 6. I wrote a scheduled pool like this static {

Re: Job is be cancelled, but the stdout log still prints

2018-03-08 Thread sundy
I got it. That’s really a big problem. Thank you very much > On 8 Mar 2018, at 21:03, kedar mhaswade wrote: > > Also, in addition to what Gary said, if you take Flink completely out of > picture and wrote a simple Java class with a main method and the static block >

What's the best way to clean up the expired rocksdb state

2018-03-11 Thread sundy
hi: my streaming application always do Key by the some keys with event timestamp, such as keyBy( “qps_1520777430”), so the expired keys(1 hours ago) are useless. And I use rocksdb to store the state, I want to know What's the best way to clean up the expired rocksdb state, should I must

Flink kafka connector with JAAS configurations crashed

2018-03-13 Thread sundy
Hi ,all I use the code below to set kafka JASS config, the serverConfig.jasspath is /data/apps/spark/kafka_client_jaas.conf, but on flink standalone deployment, it crashs. I am sure the kafka_client_jass.conf is valid, cause other applications(Spark streaming) are still working fine

Re: Watermark not worked as expected in TimeCharacteristic.EvenTime setting.

2018-03-02 Thread sundy
Interval()` to set an interval. > > Hope that helps. > > Best, > Xingcan > >> On 3 Mar 2018, at 12:08 PM, sundy <543950...@qq.com >> <mailto:543950...@qq.com>> wrote: >> >> >> >> Hi, I got a problem in Flink and need your help

Watermark not worked as expected in TimeCharacteristic.EvenTime setting.

2018-03-02 Thread sundy
Hi, I got a problem in Flink and need your help. I tried to use TimeCharacteristic.EvenTime, but the sink function never be executed. public class StreamingJob { public static void main(String[] args) throws Exception { // set up the streaming execution environment final

Re: Watermark not worked as expected in TimeCharacteristic.EvenTime setting.

2018-03-03 Thread sundy
time is the getWaterMark function called? After the keyBy operation or after the source operation? > On 3 Mar 2018, at 15:28, Hequn Cheng <chenghe...@gmail.com> wrote: > > Hi sundy, > > The default parallelism is 4. It seems that your configuration does not take

Re: Watermark not worked as expected in TimeCharacteristic.EvenTime setting.

2018-03-03 Thread sundy
Thanks a lot, use env.setParallelism(1) before the source define works (I set it before the env.execute, so it did not take effect). > On 3 Mar 2018, at 16:02, sundy <543950...@qq.com> wrote: > > Hi Hequn Cheng, > > Finally I got the problem and find the way to define