Thanks so much Vinoth. I will follow up with the links information.Cheers
From: Vinoth Chandar <[email protected]>
To: [email protected]; amir bahmanyari <[email protected]>
Sent: Thursday, October 27, 2016 5:26 PM
Subject: Re: KafkaIO()/Kafka Data Monitoring
I assume Flink uses the Kafka Simple Consumer. So, the built in JMX metrics
should be good enough?
https://cwiki.apache.org/confluence/display/KAFKA/Operations#Operations-Consumerstats
You can use something like https://github.com/jmxtrans/jmxtrans to report them
to graphite et al.
My 2c,
Vinoth
On Thu, Oct 27, 2016 at 4:38 PM, amir bahmanyari <[email protected]> wrote:
Hi Colleagues,Any suggestions on how to monitor the data rate coming to Beam
app (KafkaIO()) running in a Flink cluster from Kafka?The Flink dashboard
doesn't show any changes...its 0's all the time..
I thought there may be a utility/tool that I can intercept/monitor the rate at
which Kafka sends to the Beam app in Flink cluster?
Thanks+regards.
