Re: Beam runner statsd metrics

2019-11-08 Thread Kush Kumar Sharma
Hi Maximilian, I have used Beam's Metrics in our pipeline implementation but it's of no use. I just want to understand whether beam's metrics system automatically sends these counters to flink's statsd implementation. FYI I do get flink's general metrics like taskSlotsTotal, numRunningJobs, etc

Re: Beam runner statsd metrics

2019-11-08 Thread Maximilian Michels
Hi Kush, Beam has its own Metrics[1] which are reported via the Flink metric system. You may want to use those and utilize the Flink statsd reporter: https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/metrics.html#statsd-orgapacheflinkmetricsstatsdstatsdreporter Cheers, Max

Beam runner statsd metrics

2019-11-08 Thread Kush Kumar Sharma
Hi Devs! I am trying to use a *Statsd* client in beam to export some runner metrics. I am able to extract out metrics from the base application but once the job is submitted to its runner(in this case I am using *FlinkRunner*), statsd client stops working. This is a streaming job and I need to