Re: How to sleep for 1 sec and then call keyBy for partitioning

2018-05-16 Thread Vijay Balakrishnan
Hi, This worked out after looking at https://stackoverflow.com/questions/44436401/some-puzzles-for-the-operator-parallelism-in-flink?rq=1 Why cannot I use setParallelism after keyBy-is it not an operator ? DataStream cameraWithCubeDataStream = env .addSource(new

Re: How to sleep for 1 sec and then call keyBy for partitioning

2018-05-16 Thread Jörn Franke
Just some advice - do not use sleep to simulate a heavy task. Use real data or generated data to simulate. This sleep is garbage from a software quality point of view. Furthermore, it is often forgotten etc. > On 16. May 2018, at 22:32, Vijay Balakrishnan wrote: > > Hi, >

How to sleep for 1 sec and then call keyBy for partitioning

2018-05-16 Thread Vijay Balakrishnan
Hi, Newbie question - What I am trying to do is the following: CameraWithCubeSource source sends data-containing tuples of (cameraNbr,TS). 1. Need to partition data by cameraNbr. *2. Then sleep for 1 sec to simulate a heavy process in the task.* *3. Then need to partition data by TS and finally