Re: Querying consumer groups programmatically (from Golang)

2017-08-12 Thread Ian Duffy
Hi Jens, We did something similar to this at Zalando. https://github.com/zalando-incubator/remora It effectively supplies the kafka consumer group supply command as a http endpoint. On 12 August 2017 at 16:42, Subhash Sriram wrote: > Hi Jens, > > Have you looked at

Re: How to debug - NETWORK_EXCEPTION

2017-08-12 Thread Raghav
Hey Martin I am using default setting for queue.enqueue.timeout.ms. since I have not set it my Java client. Network doesn't seem to time out either. Could I be missing something else ? On Sat, Aug 12, 2017 at 5:18 AM, Martin Gainty wrote: > > >

Re: Forwarding consumer with kafka streams

2017-08-12 Thread Eno Thereska
Hi Ricardo, Kafka Streams should handle that case as well. What streams config are you using, could you share it? There is one parameter that is called “ConsumerConfig.AUTO_OFFSET_RESET_CONFIG” and by default it’s set to “earliest”. Any chance your app has changed it to “latest”? Thanks Eno

Re: Querying consumer groups programmatically (from Golang)

2017-08-12 Thread Subhash Sriram
Hi Jens, Have you looked at Burrow? https://github.com/linkedin/Burrow/blob/master/README.md Thanks, Subhash Sent from my iPhone > On Aug 12, 2017, at 8:55 AM, Jens Rantil wrote: > > Hi, > > I am one of the maintainers of prometheus-kafka-consumer-group-exporter[1],

Querying consumer groups programmatically (from Golang)

2017-08-12 Thread Jens Rantil
Hi, I am one of the maintainers of prometheus-kafka-consumer-group-exporter[1], which exports consumer group offsets and lag to Prometheus. The way we currently scrape this information is by periodically executing `kafka-consumer-groups.sh --describe` for each group and parse the output.

Re: How to debug - NETWORK_EXCEPTION

2017-08-12 Thread Martin Gainty
From: Raghav Sent: Thursday, August 10, 2017 12:51 AM To: Users; confluent-platf...@googlegroups.com Subject: How to debug - NETWORK_EXCEPTION Hi I am sending very small 32 byte message to Kafka broker in a tight loop with 250ms sleep. I