Hi Kush,

Beam has its own Metrics[1] which are reported via the Flink metric system. You may want to use those and utilize the Flink statsd reporter: https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/metrics.html#statsd-orgapacheflinkmetricsstatsdstatsdreporter

Cheers,
Max

[1] https://beam.apache.org/releases/javadoc/2.1.0/org/apache/beam/sdk/metrics/Metrics.html

On 08.11.19 11:31, Kush Kumar Sharma wrote:
Hi Devs!

I am trying to use a *Statsd* client in beam to export some runner metrics. I am able to extract out metrics from the base application but once the job is submitted to its runner(in this case I am using *FlinkRunner*), statsd client stops working. This is a streaming job and I need to extract out those metrics continuously(like how many files are getting processed, etc).

Are there any helper classes for this particular use case in beam? If not, are there any flink specific classes that I can use here? I have seen *MetricResults*(from *FlinkRunnerResult*) but it looks like I can only use that once the job is done not when in progress.

My current statsd client is https://github.com/tim-group/java-statsd-client

Regards
Kush Sharma

Reply via email to