GitHub user StephanEwen opened a pull request:
https://github.com/apache/flink/pull/1699
[FLINK-3435] Proparly separate IngestionTime and EventTime
This pull request changes the interplay between **Event Time** and
**Ingestion Time**
Prior to this pull request, both time characteristics were essentially
equal.
After this pull request:
- *Ingestion Time* auto generates ingestion timestamps and watermarks.
- *Event Time* does not generate and automatic watermarks, and assigns -1
as timestamps
- Timestamp extractors are not allowed to assign negative timestamps any
more
- If a default window operator finds a -1 event time timestamp, this
indicates that the system runs on Event Time and does not have a timestamp
assigner. This causes an exception so the program will immediately fail with an
informative error message.
- Mixing Event Time and Ingestion Time requires using Event Time and
using an included IngestionTimestampExtractor.
This also removes `ExecutionConfig.areTimestampsEnabled()` and similar
methods.
Desirable changes for future versions:
- At pre-flight time, deactivate custom timestamp assigners when using
Ingesion Time.
- At pre-flight time, check whether Event Time pipelines actually have
timestamp assigners.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/StephanEwen/incubator-flink time_interplay
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/1699.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1699
----
commit 7441934f885beecf51a71963b63b758769492573
Author: Stephan Ewen <[email protected]>
Date: 2016-02-22T16:58:07Z
[FLINK-3468] [streaming] Handle negative timestamps in timestamp assigner
operators
commit 3b54eba64deeff7da9fc4fb66e79fc53a84f41fb
Author: Stephan Ewen <[email protected]>
Date: 2016-02-22T18:22:47Z
[FLINK-3435] [streaming] EventTime window assigners error out on negative
timestamps
commit 843912a46ba0969319300fe055a8554db8f86a52
Author: Stephan Ewen <[email protected]>
Date: 2016-02-22T19:32:07Z
[hotfix] Minor cleanups and removal of unused parameter
commit 81458f5ee79aa1cc1f350a3bdb57252487d134b5
Author: Stephan Ewen <[email protected]>
Date: 2016-02-23T17:32:43Z
[FLINK-3435] [streaming] Proparly separate IngestionTime and EventTime
- IngestionTime now only auto-generates watermarks
- EventTime does not auto-generate ingestion watermarks
This also removes the "ExecutionConfig.areTimestampsEnabled()" flag.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
