I assume Flink uses the Kafka Simple Consumer. So, the built in JMX metrics should be good enough? https://cwiki.apache.org/confluence/display/KAFKA/Operations#Operations-Consumerstats
You can use something like https://github.com/jmxtrans/jmxtrans to report them to graphite et al. My 2c, Vinoth On Thu, Oct 27, 2016 at 4:38 PM, amir bahmanyari <[email protected]> wrote: > Hi Colleagues, > Any suggestions on how to monitor the data rate coming to Beam app > (KafkaIO()) running in a Flink cluster from Kafka? > The Flink dashboard doesn't show any changes...its 0's all the time.. > I thought there may be a utility/tool that I can intercept/monitor the > rate at which Kafka sends to the Beam app in Flink cluster? > > Thanks+regards. >
