[
https://issues.apache.org/jira/browse/FLINK-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15149103#comment-15149103
]
ASF GitHub Bot commented on FLINK-3379:
---------------------------------------
GitHub user StephanEwen opened a pull request:
https://github.com/apache/flink/pull/1646
[FLINK-3379] [FLINK-3415] [streaming] Refactor TimestampExtractor
This pull request divides the TimestampExtractor into two separate classes
- one class handled periodic watermarks
- the other class handled watermarks triggered by individual elements
This also makes sure that any timestamp assigner / watermark generators
cannot generate
negative watermarks.
This also adds tests for the operators that run the timestamp extractors.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/StephanEwen/incubator-flink
refactor_extractors
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/1646.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1646
----
commit 1e3c4597dd1785430735950eef2b7b8c2154faaf
Author: Stephan Ewen <[email protected]>
Date: 2016-02-16T09:37:59Z
[FLINK-3379] [FLINK-3415] [streaming] Refactor TimestampExtractor into two
separate classes
- one class handled periodic watermarks
- the other class handled watermarks triggered by elements
This also makes sure that any timestamp assigner / watermark generators
cannot generate
negative watermarks
----
> Refactor TimestampExtractor
> ---------------------------
>
> Key: FLINK-3379
> URL: https://issues.apache.org/jira/browse/FLINK-3379
> Project: Flink
> Issue Type: Improvement
> Components: Streaming
> Affects Versions: 0.10.1
> Reporter: Stephan Ewen
> Assignee: Stephan Ewen
> Priority: Critical
> Fix For: 1.0.0
>
>
> Based on a lot of user feedback, the current {{TimestampExtractor}} seems
> very confusing. It implements simultaneously two modes of generating
> watermarks:
> - Each record that passes through can decide to cause a watermark.
> - The timestamp extractor can define a certain watermark timestamp which is
> periodically picked up by the system and triggers a watermark (if larger than
> the previous watermark).
> Figuring out how these modes interplay, and how to define the methods to only
> use one mode has been quite an obstacle for several users. We should break
> this class into two different classes, one per mode of generating watermarks,
> to make it easier to understand.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
