Boyuan Zhang created BEAM-9312:
----------------------------------
Summary: Make MonotonicWatermarkEstimator configurable for whether
to ignore late timestamp.
Key: BEAM-9312
URL: https://issues.apache.org/jira/browse/BEAM-9312
Project: Beam
Issue Type: Improvement
Components: sdk-py-core, sdk-py-harness
Reporter: Boyuan Zhang
Current implementation of MonotonicWatermarkEstimator throws error and stop the
pipeline when there is a late timestamp. But there are more potential options
like:
(1) Suppress the error and emit the item as possibly late data.
(2) Move the timestamp forward to respect the watermark.
We should consider making MonotonicWatermarkEstimator configurable with these
options, or providing different types of MonotonicWatermarkEstimator to
handle different options.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
