[
https://issues.apache.org/jira/browse/BEAM-2456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jean-Baptiste Onofré updated BEAM-2456:
---------------------------------------
Summary: Introduce generic metric poll thread interceptor & generic sink
(was: Introduce generic metric poll thread interceptor)
> Introduce generic metric poll thread interceptor & generic sink
> ---------------------------------------------------------------
>
> Key: BEAM-2456
> URL: https://issues.apache.org/jira/browse/BEAM-2456
> Project: Beam
> Issue Type: New Feature
> Components: sdk-ideas
> Reporter: Jean-Baptiste Onofré
> Assignee: Jean-Baptiste Onofré
>
> The Spark runner provides a convenient feature which is the metric sink.
> By configuration, it allows us to configure a metric sink using
> {{metrics.properties}} configuration containing:
> {code}
> driver.sink.graphite.class=org.apache.beam.runners.spark.metrics.sink.GraphiteSink
> driver.sink.graphite.host=localhost
> driver.sink.graphite.port=2003
> driver.sink.graphite.prefix=spark
> driver.sink.graphite.period=1
> driver.sink.graphite.unit=SECONDS
> {code}
> This approach is very convenient to send the metric to any sink. I think we
> can apply this logic in generic way working with any runner.
> The idea is to use a metric poll thread in the pipeline (that we can enable
> via {{PipelineOptions}}) and send to a sink.
> I started a PoC about that.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
