Elias Levy created FLINK-6472:
---------------------------------
Summary: BoundedOutOfOrdernessTimestampExtractor does not bound
out of orderliness
Key: FLINK-6472
URL: https://issues.apache.org/jira/browse/FLINK-6472
Project: Flink
Issue Type: Bug
Components: Streaming
Affects Versions: 1.3.0
Reporter: Elias Levy
{{BoundedOutOfOrdernessTimestampExtractor}} attempts to emit watermarks that
lag behind the largest observed timestamp by a configurable time delta. It
fails to so in some circumstances.
The class extends {{AssignerWithPeriodicWatermarks}}, which generates
watermarks in periodic intervals. The timer for this intervals is a processing
time timer.
In circumstances where there is a rush of events (restarting Flink, unpausing
an upstream producer, loading events from a file, etc), many events with
timestamps much larger that what the configured bound would normally allow will
be sent downstream without a watermark. This can have negative effects
downstream, as operators may be buffering the events waiting for a watermark to
process them, thus leading the memory growth and possible out-of-memory
conditions.
It is probably best to have a bounded out of orderliness extractor that is
based on the punctuated timestamp extractor, so we can ensure that watermarks
are generated in a timely fashion in event time, with the addition of process
time timer to generate a watermark if there has been a lull in events, thus
also bounding the delay of generating a watermark in processing time.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
