[
https://issues.apache.org/jira/browse/BEAM-1941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jingsong Lee reassigned BEAM-1941:
----------------------------------
Assignee: (was: Jingsong Lee)
> Add Watermark Metrics in Runners
> --------------------------------
>
> Key: BEAM-1941
> URL: https://issues.apache.org/jira/browse/BEAM-1941
> Project: Beam
> Issue Type: New Feature
> Components: runner-ideas
> Reporter: Jingsong Lee
> Priority: Major
>
> The source watermark metrics show the consumer latency of Source.
> It allows the user to know the health of the job, or it can be used to
> monitor and alarm.
> Since each runner is likely already tracking a watermark, another option here
> is to just have the runner report it appropriately, rather than having the
> source report it using metrics. This also addresses the fact that even if the
> source has advanced to 8:00, the runner may still know about buffered
> elements at 7:00, and so not advance the watermark all the way to 8:00.
> [~bchambers]
> Includes:
> 1.Source watermark (`min` amongst all splits):
> type = Gauge, namespace = io, name = source_watermark
> 2.Source watermark per split:
> type = Gauge, namespace = io.splits, name = <split_id>.source_watermark
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
