Re: "Futures timed out" when trying to cancel a job with savepoint

2018-07-29 Thread vino yang
Hi Julio, We also encountered this problem on YARN, Savepoint has been completed, and JM has been successfully stopped, but the client is still trying to access the original JM port, which caused a timeout. It seems that this is a problem with Flink itself. I can't give you the answer to this

Re: not found: type CustomWatermarkEmitter

2018-07-29 Thread vino yang
Hi Mich, These two mistakes are obvious. 1): The compiler can not find the definition of CustomWatermarkEmitter. Did you define it? Or import the dependency if it defines in other place? 2): The type of variable "myCustomer" is "DataStreamSource", but env.addSource method receive a source

Re: Order of events in a Keyed Stream

2018-07-29 Thread Renjie Liu
Hi, Another way to ensure order is by adding a logical version number for each message so that earlier version will not override later version. Timestamp depends on your ntp server works correctly. On Sun, Jul 29, 2018 at 3:52 PM Niels Basjes wrote: > Hi, > > The basic thing is that you will

Re: not found: type CustomWatermarkEmitter

2018-07-29 Thread Renjie Liu
It seems that it's related with your development environment settings. On Sun, Jul 29, 2018 at 4:39 PM Mich Talebzadeh wrote: > Hi, > > I have written a simple test program as below > > import java.util.Properties > import java.util.Arrays > import

not found: type CustomWatermarkEmitter

2018-07-29 Thread Mich Talebzadeh
Hi, I have written a simple test program as below import java.util.Properties import java.util.Arrays import org.apache.flink.api.common.functions.MapFunction import org.apache.flink.api.java.utils.ParameterTool import org.apache.flink.streaming.api.datastream.DataStream import

Re: Order of events in a Keyed Stream

2018-07-29 Thread Niels Basjes
Hi, The basic thing is that you will only get the messages in a guaranteed order if the order is maintained in all steps from creation to use. In Kafka order is only guaranteed for messages in the same partition. So if you need them in order by account then the producing system must use the

Re: Order of events in a Keyed Stream

2018-07-29 Thread Congxian Qiu
Hi, Maybe the messages of the same key should be in the *same partition* of Kafka topic 2018-07-29 11:01 GMT+08:00 Hequn Cheng : > Hi harshvardhan, > If 1.the messages exist on the same topic and 2.there are no rebalance and > 3.keyby on the same field with same value, the answer is yes. > >