Re: Order events by filed that does not represent time

2019-12-11 Thread David Anderson
Krzysztof, Note that if you want to have Flink treat these sequence numbers as event time timestamps, you probably can, so long as they are generally increasing, and there's some bound on how out-of-order they can be. The advantage to doing this is that you might be able to use Flink SQL's event

Re: Order events by filed that does not represent time

2019-12-11 Thread Timo Walther
Hi Krzysztof, first of all Flink does not sort events based on timestamp. The concept of watermarks just postpones the triggering of a time operation until the watermark says all events until a time t have arrived. For your problem, you can simply use a ProcessFunction and buffer the events

Order events by filed that does not represent time

2019-12-10 Thread KristoffSC
Hi, Is it possible to use an field that does not represent timestamp to order events in Flink's pipeline? In other words, I will receive a stream of events that will ha a sequence number (gaps are possible). Can I maintain the order of those events based on this field same as I would do for time