To clarify one thing: keep in mind that Flink does not support per-key watermarks. Watermarks are typically generated per-source, or in the case of kafka, can be per-partition. An idle source (or in the case of kafka, an idle partition) can prevent an event-time window from being triggered, but you don't need to worry about this on a per-key basis.
On Fri, Sep 14, 2018 at 6:08 AM Taher Koitawala <taher.koitaw...@gslab.com> wrote: > Yes in many cases what we have faced that let's say in a keyed stream an > element of a specific key comes in which triggers a new window. If a > corresponding elements of the same key does not arrive a new watermark is > not generated for the window to purge. Then we faced issues with flink > keeping records in the window and not purging them at all. For this reason > we explicitly had to write some logic to generate a timestamp this is > greater than the currentTimestamp and then forcefully make the window purge > its elements. > > > > On 14-Sep-2018 7:48 AM, "vino yang" <yanghua1...@gmail.com> wrote: > > Hi Taher, > > For some questions, I suggest you read the documentation related to Flink > EventTime first, for example [1] > > About this question: > What happens if the watermark is same as the timestamp? > > Here "timestamp", do you mean the current timestamp of Processing time? If > that's the best, it's an ideal state, and it's often difficult to achieve > this state, because eventtime always has more or less delay as events are > transmitted from the source to the processing system. > > Thanks, vino. > > [1]: > https://ci.apache.org/projects/flink/flink-docs-release-1.6/dev/event_timestamps_watermarks.html > > Taher Koitawala <taher.koitaw...@gslab.com> 于2018年9月14日周五 上午2:07写道： > >> Hi All, >> Can someone show a good example of how watermarks need to be >> generated when using EventTime windows? What happens if the watermark is >> same as the timestamp? How does the watermark help in the window to be >> triggered and what if watermarks are kept behind the currentTimestamps in >> the assignTimestampsAndWatermarks function. >> >> >> > -- *David Anderson* | Training Coordinator | data Artisans -- Join Flink Forward - The Apache Flink Conference Stream Processing | Event Driven | Real Time
