Hi Jens,

Have you looked at Burrow?



Sent from my iPhone

> On Aug 12, 2017, at 8:55 AM, Jens Rantil <jens.ran...@gmail.com> wrote:
> Hi,
> I am one of the maintainers of prometheus-kafka-consumer-group-exporter[1],
> which exports consumer group offsets and lag to Prometheus. The way we
> currently scrape this information is by periodically executing
> `kafka-consumer-groups.sh --describe` for each group and parse the output.
> Recently the output from `kafka-consumer-groups.sh --describe` was
> changed[2]. While I am working on a patch[3] to accomodate for the new
> output format I was wondering if there is an easier, possibly more
> stable[4] and more future proof, way for our project to extract the
> information we are interested in. Does anyone know of a Go library that
> could extract the metrics we need? Or would it make sense to refactor
> `kafka-consumer-groups.sh` to support a more structured output? I'd love to
> hear your input.
> Also, if Kafka exported the same metrics through JMX our project would not
> exist, but maybe that's another story...
> Cheers,
> Jens
> [1] https://github.com/kawamuray/prometheus-kafka-consumer-group-exporter
> [2]
> https://github.com/kawamuray/prometheus-kafka-consumer-group-exporter/issues/24
> [3]
> https://github.com/kawamuray/prometheus-kafka-consumer-group-exporter/pull/29
> [4] We've also encountered `kafka-consumer-groups.sh` hanging a few times
> in production. There's a race condition somewhere in the script, most
> likely when a topic is rebalancing. Currently we kill the process if it
> doesn't finish within a timeout. See
> https://github.com/kawamuray/prometheus-kafka-consumer-group-exporter/blob/e4cdc3b1245f636d89d7e227066f02578d732165/kafka/collector.go#L44
> .
> -- 
> Want to communicate with me securely? You can find my PGP key here
> <https://pgp.mit.edu/pks/lookup?op=get&search=0xBA632DB691DA03B8>.

Reply via email to