[
https://issues.apache.org/jira/browse/BEAM-1941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jingsong Lee reassigned BEAM-1941:
----------------------------------
Assignee: Jingsong Lee
> Add Source Watermark Metrics in Runners
> ---------------------------------------
>
> Key: BEAM-1941
> URL: https://issues.apache.org/jira/browse/BEAM-1941
> Project: Beam
> Issue Type: New Feature
> Components: runner-ideas
> Reporter: Jingsong Lee
> Assignee: Jingsong Lee
>
> The source watermark metrics show the consumer latency of Source.
> It allows the user to know the health of the job, or it can be used to
> monitor and alarm.
> Since each runner is likely already tracking a watermark, another option here
> is to just have the runner report it appropriately, rather than having the
> source report it using metrics. This also addresses the fact that even if the
> source has advanced to 8:00, the runner may still know about buffered
> elements at 7:00, and so not advance the watermark all the way to 8:00.
> [~bchambers]
> Includes:
> 1.Source watermark (`min` amongst all splits):
> type = Gauge, namespace = io, name = source_watermark
> 2.Source watermark per split:
> type = Gauge, namespace = io.splits, name = <split_id>.source_watermark
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
