Hi Kush,
Beam has its own Metrics[1] which are reported via the Flink metric
system. You may want to use those and utilize the Flink statsd reporter:
https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/metrics.html#statsd-orgapacheflinkmetricsstatsdstatsdreporter
Cheers,
Max
[1]
https://beam.apache.org/releases/javadoc/2.1.0/org/apache/beam/sdk/metrics/Metrics.html
On 08.11.19 11:31, Kush Kumar Sharma wrote:
Hi Devs!
I am trying to use a *Statsd* client in beam to export some runner
metrics. I am able to extract out metrics from the base application but
once the job is submitted to its runner(in this case I am using
*FlinkRunner*), statsd client stops working. This is a streaming job and
I need to extract out those metrics continuously(like how many files are
getting processed, etc).
Are there any helper classes for this particular use case in beam? If
not, are there any flink specific classes that I can use here? I have
seen *MetricResults*(from *FlinkRunnerResult*) but it looks like I can
only use that once the job is done not when in progress.
My current statsd client is https://github.com/tim-group/java-statsd-client
Regards
Kush Sharma