Re: High CPU usage for idle kafka server

2015-06-05 Thread Jiangjie Qin
Has this to do with KAFKA-1461? Can you see which thread is taking a lot of cpu? Some jconsole plugin can get that information. Jiangjie (Becket) Qin On 6/5/15, 2:57 PM, "pundlik.anuja" wrote: >Hi Jay, > >Good to hear from you. I met you at the kafka meetup at linkedin. > >- No, I am running ka

Re: High CPU usage for idle kafka server

2015-06-05 Thread pundlik.anuja
Hi Jay, Good to hear from you. I met you at the kafka meetup at linkedin. - No, I am running kafka_2.11-0.8.2.1 Are there any logs/ any info that I can provide that will help you understand what could be the issue? Thanks, Anuja On Fri, Jun 5, 2015 at 2:36 PM, Jay Kreps wrote: > This sounds

Re: High CPU usage for idle kafka server

2015-06-05 Thread Jay Kreps
This sounds a lot like a bug we fixed in 0.8.2.0, no chance you are running that pre-release version is there? -Jay On Wed, Jun 3, 2015 at 9:43 PM, Anuja Pundlik (apundlik) wrote: > Hi, > > I am using Kafka 0.8.2.1. > We have 1 zookeeper, 3 kafka brokers. > We have 9 topics, out of which 1 topi

Re: High CPU usage for idle kafka server

2015-06-05 Thread pundlik.anuja
There are no messages being sent or received. The system is idle. however, there seems to be some GC going on in the kafka broker and some socket reads and writes. It is using approx 500MB of memory. On Fri, Jun 5, 2015 at 1:44 PM, pundlik.anuja wrote: > Hi Otis, > How do I check garbage collect

Re: High CPU usage for idle kafka server

2015-06-05 Thread pundlik.anuja
Hi Otis, How do I check garbage collection on kafka broker? On Thu, Jun 4, 2015 at 1:24 PM, Otis Gospodnetic wrote: > How's their garbage collection doing? > > Otis > -- > Monitoring * Alerting * Anomaly Detection * Centralized Log Management > Solr & Elasticsearch Support * http://sematext.com

Query on kafka topic metadata

2015-06-05 Thread Pavan Chenduluru
Hi, I am new to kafka and I have a doubt. How to read specified topic statistics from kafka server? I want to read below parameters about existing topic from kafka. 1) How many activeMessages 2) How many activeSubscriptions 3) How many totalMessages 4) How many totalSubscriptions 5) How mnay de

Re: Consumer lag lies - orphaned offsets?

2015-06-05 Thread Joel Koshy
On Fri, Jun 05, 2015 at 12:53:00AM -0400, Otis Gospodnetić wrote: > Hi Joel, > > On Thu, Jun 4, 2015 at 8:52 PM, Joel Koshy wrote: > > > Hi Otis, > > > > Yes this is a limitation in the old consumer. i.e., a number of > > per-topic/partition mbeans remain even on a rebalance. Those need to > > b

Re: Multiple instances of HL Consumer

2015-06-05 Thread Sharninder Khera
You can have the same consumer id and Kafka will balance partitions across the two instances automatically. When one of them dies the partitions are rebalanced and assigned to the remaining alive consumers.  _ From: Panda, Samaresh Sent: Friday, June 5, 2015 7:

Multiple instances of HL Consumer

2015-06-05 Thread Panda, Samaresh
I've a HL consumer receiving messages using four threads (four partitions). This is a stand-alone Java client. For fail-safe reasons, I want to run another instance of the exact same Java client in a different box. Here are my questions: > Can I keep the same consumer group name or it must be d