[
https://issues.apache.org/jira/browse/BEAM-1941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kenneth Knowles updated BEAM-1941:
----------------------------------
This Jira ticket has a pull request attached to it, but is still open. Did the
pull request resolve the issue? If so, could you please mark it resolved? This
will help the project have a clear view of its open issues.
> Add Watermark Metrics in Runners
> --------------------------------
>
> Key: BEAM-1941
> URL: https://issues.apache.org/jira/browse/BEAM-1941
> Project: Beam
> Issue Type: New Feature
> Components: runner-ideas
> Reporter: Jingsong Lee
> Priority: P3
>
> The source watermark metrics show the consumer latency of Source.
> It allows the user to know the health of the job, or it can be used to
> monitor and alarm.
> Since each runner is likely already tracking a watermark, another option here
> is to just have the runner report it appropriately, rather than having the
> source report it using metrics. This also addresses the fact that even if the
> source has advanced to 8:00, the runner may still know about buffered
> elements at 7:00, and so not advance the watermark all the way to 8:00.
> [~bchambers]
> Includes:
> 1.Source watermark (`min` amongst all splits):
> type = Gauge, namespace = io, name = source_watermark
> 2.Source watermark per split:
> type = Gauge, namespace = io.splits, name = <split_id>.source_watermark
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
