To subscribe

Dear Apache Kafka, I have couple of questions on Kafka server cluster. I am working on Kafka project and maintaining Kafka brokers are too difficult due to continues failures. Please add me to subscribe Apache Kafka user list. Thanks, Rafeeq S (Inspire to work on Kafka)

What information does FetchRequestAndResponseMetrics name contain?

Hello everyone, We are adding Kafka 0.8.x monitoring support to SPM http://sematext.com/spm/ here at Sematex and I have one question about JMX Kafka beans: In Jconsole for my local setup I see: kafka.consumer:type=FetchRequestAndResponseMetrics,name=

Fwd: How to parse some of JMX Bean's names.

Hello everyone, We are adding Kafka 0.8.x monitoring support to SPM http://sematext.com/spm/ here at Sematext. Unfortunately, we quickly hit an issue caused by the new bean naming convention that embeds things like topic and host names in the beans along with metrics, separated by dashes, making

Re: MetricId can be EMPTY in ProducerRequestMetrics

You will need to set the client.id property in the producer. Thanks, Jun On Mon, Jun 2, 2014 at 5:12 AM, Vladimir Tretyakov vladimir.tretya...@sematext.com wrote: Hello everyone, We are adding Kafka 0.8.x monitoring support to SPM http://sematext.com/spm/ here at Sematex and I have one

Re: CreateMessageStream API

Pradeep, Kafka guarantees ordering of data per partition. So if you have several producers that send data with a key, you will have all messages for the same key ordered in a single partition. If you attach a consumer to one partition, you can then consume data in order per key. It will help to

Re: What information does FetchRequestAndResponseMetrics name contain?

fcaaea29 is some sort of uuid. 0-0 is clientid-brokerId (to fetch from). Thanks, Jun On Mon, Jun 2, 2014 at 5:16 AM, Vladimir Tretyakov vladimir.tretya...@sematext.com wrote: Hello everyone, We are adding Kafka 0.8.x monitoring support to SPM http://sematext.com/spm/ here at Sematex and

Re: What information does FetchRequestAndResponseMetrics name contain?

Hi Jun Rao, thx for quick answer. That means I can see at the same time values like : af-servers-ConsumerFetcherThread-af-servers-wawanawna-Dell-1401353748289- *fcaaea28*-0-0-host_localhost-my-port_9092-FetchRequestRateAndTimeMs and

Re: MetricId can be EMPTY in ProducerRequestMetrics

Aha, I see, thx a lot. On Mon, Jun 2, 2014 at 7:51 PM, Jun Rao jun...@gmail.com wrote: You will need to set the client.id property in the producer. Thanks, Jun On Mon, Jun 2, 2014 at 5:12 AM, Vladimir Tretyakov vladimir.tretya...@sematext.com wrote: Hello everyone, We are

Re: Kafka Broker frequent shutdown with exception

Hi Rafeeq, With respect to question #2, are you stopping brokers using kafka-server-stop.sh and are they all on a single host? If so, the script finds anything that looks like a Kafka server and should knock out all the brokers at once. If your cluster runs across multiple hosts something else

log retention and rollover

This might be a bit unusual. We have a topic that we only need to keep last 5 minutes of msgs so that replay from beginning is fast. Although retention.ms has time unit of minute, segment.ms ONLY has time unit of hour. If I understand cleanup correctly, it can only delete files that are rolled

Re: Add Partition in 0.7.1

Yes, the producers do not need to be bounced, they will pick up the updated partition info automatically. Guozhang On Mon, Jun 2, 2014 at 5:54 AM, Surendranauth Hiraman suren.hira...@velos.io wrote: To confirm, the producers do not need to be bounced in this case as well? -Suren On

Re: How to parse some of JMX Bean's names.

That is indeed a problem, for now, we recommend group name and topic names to use _ when there is a need for -, but this should be fixed systematically. For you use case, could you change your topic/group name using _? Also, do you mind to file a JIRA ticket to keep track of this issue? Guozhang

Re: log retention and rollover

Steven, We initially set the rolling criterion based on hours to avoid too frequent log rolling and in turn too small segment files. For your case this may be reasonable to set the rolling criterion on minutes. Could you file a JIRA? Guozhang On Mon, Jun 2, 2014 at 4:00 PM, Steven Wu

Re: log retention and rollover

created KAFKA-1480 https://issues.apache.org/jira/browse/KAFKA-1480. Thanks! it's also generally better to have consistent/matching time unit for these two configs. On Mon, Jun 2, 2014 at 4:22 PM, Guozhang Wang wangg...@gmail.com wrote: Steven, We initially set the rolling criterion based

Re: How to parse some of JMX Bean's names.

Hi Guozhang, On Mon, Jun 2, 2014 at 7:18 PM, Guozhang Wang wangg...@gmail.com wrote: That is indeed a problem, for now, we recommend group name and topic names to use _ when there is a need for -, but this should be fixed systematically. Right! For you use case, could you change your

Re: mBean to monitor message per partitions in topic

Or if you want something more user-friendly you can try out https://github.com/quantifind/KafkaOffsetMonitor Claude On Sat, May 31, 2014 at 2:40 AM, Jun Rao jun...@gmail.com wrote: Ok, if you want to track this on the broker, you can use the consumer offset check tool that Guozhang

Re: What information does FetchRequestAndResponseMetrics name contain?

Think of UUID as some random bytes. Thanks, Jun On Mon, Jun 2, 2014 at 12:01 PM, Vladimir Tretyakov vladimir.tretya...@sematext.com wrote: Hi Jun Rao, thx for quick answer. That means I can see at the same time values like :