[
https://issues.apache.org/jira/browse/FLINK-4812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16268547#comment-16268547
]
ASF GitHub Bot commented on FLINK-4812:
---------------------------------------
GitHub user zentol opened a pull request:
https://github.com/apache/flink/pull/5092
[FLINK-4812][metrics] Expose currentLowWatermark for all operators
## What is the purpose of the change
This PR makes all operators expose the current low watermark through the
metric system. As mentioned in the documentation, the currentLowWatermark is
the lowest watermark an operator has received, with the exception for sources
and watermark assigners for which it is the lowest emitted watermark.
## Brief change log
* remove watermark metric logic from `Stream[Two]InputProcessor`
* this implies that this metric is no longer measured at the task level
at all
* introduce `WatermarkGauge` class to decouple metric from local state of
operator classes (i.e. some currentLowWatermark field)
* measure the lowest watermark received by operators in
* `AbstractStreamOperator#processWatermark`
* `AsyncWaitOperator#processWatermark`
* measure the lowest watermark emitted by sources their `SourceContext`
* measure the lowest watermark emitted by
`TimestampsAndPeriodicWatermarksOperator` and
`TimestampsAndPunctuatedWatermarksOperator`
* update metrics reference in the documentation
## Verifying this change
This change modified the following tests:
*
`TimestampsAndPunctuatedWatermarksOperatorTest#testTimestampsAndPeriodicWatermarksOperator`
*
`TimestampsAndPeriodicWatermarksOperatorTest#testTimestampsAndPeriodicWatermarksOperator`
* `StreamSourceOperatorTest#testAutomaticWatermarkContext`
* `AsyncWaitOperatorTest#testEventTime`
This change added the following tests:
* `WatermarkGaugeTest`
* `StreamOperatorWatermarkMetricsTest`
* `StreamSourceOperatorTest#testManualWatermarkContextWatermarkMetric`
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: (no)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (**yes**)
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
- The S3 file system connector: (no)
## Documentation
- Does this pull request introduce a new feature? (no)
- If yes, how is the feature documented? (not applicable)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zentol/flink 4812
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/5092.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5092
----
commit 5970e438778f3dc86a42250c413b6fca3ca884ff
Author: zentol <[email protected]>
Date: 2017-11-27T14:55:46Z
[FLINK-4812][metrics] Expose currentLowWatermark for all operators
----
> Report Watermark metrics in all operators
> -----------------------------------------
>
> Key: FLINK-4812
> URL: https://issues.apache.org/jira/browse/FLINK-4812
> Project: Flink
> Issue Type: Improvement
> Components: Metrics
> Reporter: Robert Metzger
> Assignee: Chesnay Schepler
> Priority: Critical
> Fix For: 1.5.0
>
>
> As reported by a user, Flink does currently not export the current low
> watermark for sources
> (http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/currentLowWatermark-metric-not-reported-for-all-tasks-td13770.html).
> This JIRA is for adding such a metric for the sources as well.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
