Re: Events can overtake watermarks

2018-07-23 Thread Gyula Fóra
Yea, now that I think about it, thats probably the case. Sorry to bother :) Gyula Gyula Fóra ezt írta (időpont: 2018. júl. 23., H, 11:04): > Hm I wonder it could be because the downstream operator is a 2 input > operator and I do some filtering on the source elements to direct to one of > the

Re: Events can overtake watermarks

2018-07-23 Thread Gyula Fóra
Hm I wonder it could be because the downstream operator is a 2 input operator and I do some filtering on the source elements to direct to one of the inputs. The filtering logic is chained but I guess in this case it can happen that the downstream operators reads 2 events from one input channel

Re: Events can overtake watermarks

2018-07-23 Thread Gyula Fóra
Hi guys, Let me clarify. There is a single source with parallelism 1 and a single downstream operator with parallelism > 1. So the watermark is strictly controlled by the source. Also I am talking about calls to the processWatermark function of the downstream operator not about the watermark

Re: Events can overtake watermarks

2018-07-23 Thread Stefan Richter
Hi, events overtaking watermarks doesn’t sound like a „wrong“ behaviour, only watermarks overtaking events would be bad. Do you think this only stated from Flink 1.5? To me this does not sound like a problem, but not sure if it is intended. Looping in Aljoscha, just in case. Best, Stefan >

Re: Events can overtake watermarks

2018-07-23 Thread Aljoscha Krettek
Hi, this happens only with Flink 1.5.1? I would expect that in a parallel setting it can happen that a watermark is delayed downstream because the watermark from the other inputs has not caught up yet to "watermark1" (because the watermark at an operator is the minimum of the input

Events can overtake watermarks

2018-07-22 Thread Gyula Fóra
Hi, In 1.5.1 I have noticed some strange behaviour that happens quite frequently and I just want to double check with you that this is intended. If I have a non-parallel source that takes the following actions: emit: event1 emit: watermark1 emit: event2 it can happen that a downstream operators