On Thu, 17 Jan 2019 at 00:44, Peter Bukowinski <pmb...@gmail.com> wrote:
> On each broker, we have a process (scheduled with cron) that polls the > kafka jmx api every 60 seconds. It sends the metrics data to graphite ( > https://graphiteapp.org). We have graphite configured as a data source > for grafana (https://grafana.com) and use it to build various dashboards > to present the metrics we’re interested in. > > There are various jmx-to-graphite tools available. We use one written in > house, but this one looks like it’ll do the job: > https://github.com/logzio/jmx2graphite Hi Peter, We use this reporter (https://github.com/damienclaveau/kafka-graphite), which we add to the classpath and configure it in Kafka's server.properties: kafka.metrics.reporters=com.criteo.kafka.KafkaGraphiteMetricsReporter kafka.metrics.polling.interval.secs=60 kafka.graphite.metrics.reporter.enabled=true kafka.graphite.metrics.host=carbon.service.consul kafka.graphite.metrics.port=2003 kafka.graphite.metrics.group={{ grains['fqdn']|replace(".","_") }} kafka.graphite.metrics.exclude.regex=(kafka.network.*|kafka.log.*|kafka.cluster.*(InSyncReplicasCount|ReplicasCount|UnderReplicated)) kafka.graphite.dimension.enabled.meanRate=false kafka.graphite.dimension.enabled.rate1m=false kafka.graphite.dimension.enabled.rate5m=false kafka.graphite.dimension.enabled.rate15m=false kafka.graphite.dimension.enabled.min=false kafka.graphite.dimension.enabled.max=false kafka.graphite.dimension.enabled.mean=false kafka.graphite.dimension.enabled.sum=false kafka.graphite.dimension.enabled.stddev=false kafka.graphite.dimension.enabled.median=false kafka.graphite.dimension.enabled.p75=false kafka.graphite.dimension.enabled.p95=false kafka.graphite.dimension.enabled.p98=false kafka.graphite.dimension.enabled.p99=false kafka.graphite.dimension.enabled.p999=false That spares you from having to run a cronjob and a jmx bridge. It works also with Kafka 1.x and 2.x -- Gabriele