[
https://issues.apache.org/jira/browse/BEAM-1941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17546493#comment-17546493
]
Kenneth Knowles commented on BEAM-1941:
---------------------------------------
This issue has been migrated to https://github.com/apache/beam/issues/18359
> Add Watermark Metrics in Runners
> --------------------------------
>
> Key: BEAM-1941
> URL: https://issues.apache.org/jira/browse/BEAM-1941
> Project: Beam
> Issue Type: New Feature
> Components: runner-ideas
> Reporter: Jingsong Lee
> Priority: P3
>
> The source watermark metrics show the consumer latency of Source.
> It allows the user to know the health of the job, or it can be used to
> monitor and alarm.
> Since each runner is likely already tracking a watermark, another option here
> is to just have the runner report it appropriately, rather than having the
> source report it using metrics. This also addresses the fact that even if the
> source has advanced to 8:00, the runner may still know about buffered
> elements at 7:00, and so not advance the watermark all the way to 8:00.
> [~bchambers]
> Includes:
> 1.Source watermark (`min` amongst all splits):
> type = Gauge, namespace = io, name = source_watermark
> 2.Source watermark per split:
> type = Gauge, namespace = io.splits, name = <split_id>.source_watermark
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
