Re: FlinkRunner Graphite metrics

2020-09-15 Thread Maximilian Michels
Which metrics specifically do you mean? Beam metrics (e.g. backlogBytes) or Flink metrics (e.g. numRecordsOut)? Quick look at the code (ReaderInvocationUtil) reveals that the Beam metrics should be reported correctly. The Flink native metrics are always reported, independently of the type of

FlinkRunner Graphite metrics

2020-09-08 Thread Binh Nguyen Van
Hi, I have a streaming pipeline that reads messages from Kafka (KafkaIO.read), transforms, and then writes to Kafka (KafkaIO.write). It is using Apache Beam 2.23.0 and Flink 1.10.1. The pipeline is working as expected and I am trying to collect some metrics through Graphite so I did the following