Thanks Till - This is helpful to know. Sameer
On Tue, Oct 11, 2016 at 12:20 PM, Till Rohrmann <trohrm...@apache.org> wrote: > Hi Sameer, > > the CEP operator will take care of ordering the elements. > > Internally what happens is that the elements are buffered before being > applied to the state machine. The operator only applies the elements after > it has seen a watermark which is greater than the timestamps of the > elements being applied to the NFA. Since the elements are kept in a > priority queue wrt the timestamp they will be in order. > > Cheers, > Till > > On Tue, Oct 11, 2016 at 1:51 PM, Sameer W <sam...@axiomine.com> wrote: > >> Hi, >> >> If using CEP with event-time I have events which can be slightly out of >> order and I want to sort them by timestamp within their time-windows before >> applying CEP- >> >> For example, if using 5 second windows and I use the following >> >> ds2 = ds.keyBy.window(TumblingWindow(10 seconds).apply(/*Sort by >> Timestamp*/); >> >> Next assign watermarks again on ds2 (because elements in ds2 will all >> have the same timestamp of WINDOW_END_TIME-1ms) >> ds2.assignTimestampsAndWatermarks() >> >> Finally apply CEP on ds2 with a WITHIN window of 5 seconds (shorter >> timestamp than the one I used earlier). >> >> The reasoning is, if I am using the next() operator in CEP, the events >> should be in the order of their timestamps. >> >> Is this the right way to handle this problem? I have heard people say >> that assigning watermarks twice can lead to wrong results. But don't I need >> to assign timestamps once more in this scenario. >> >> Thanks, >> Sameer >> >> >> >> >> >
