Re: Kafka Monitoring

You can look at enabling JMX on kafka ( https://stackoverflow.com/questions/36708384/enable-jmx-on-kafka-brokers) using JMXTrans (https://github.com/jmxtrans/jmxtrans) and a config ( https://github.com/wikimedia/puppet-kafka/blob/master/kafka-jmxtrans.json.md) to gather stats, and insert them into

Re: difficulty to delete a topic because of its syntax

You *could* go in to zookeeper and nuke the topic, then delete the files on disk Slightly more risky but it should work On Wednesday, 5 October 2016, Manikumar wrote: > Kafka doesn't support white spaces in topic names. Only support '.', '_' > and '-' these are

Re: What communication is happening between Kafka brokers?

There will also be inter-broker replication traffic, and controller communications (the controller runs on an elected broker in the cluster). If you're using security features in Kafka 0.9, you may see additional auth traffic between brokers. That's all I can think of off the top of my head.

Re: New broker ignoring retention

FWIW, we've had good luck changing the mtime. No problems found. On Mon, Apr 6, 2015 at 4:37 PM, Todd Palino tpal...@gmail.com wrote: I answered this in IRC, but the issue is that retention depends on the modification time of the log segments on disk. When you copy a partition from one broker

Monitor data to missing topic

Hi all, I'm trying to find an existing metric, or method to monitor for data being sent to a topic that doesn't exist. Is there a JMX stat I should look at now, or some other way you catch this sort of thing? We'd like to get ahead of our users noticing a missing topic, or sending data to a

Tuning replication

Good day all, We're running a good sized Kafka cluster, running 0.8.1, and during our peak traffic times replication falls behind. I've been doing some reading about parameters for tuning replication, but I'd love some real world experience and insight. Some general questions: * Does Kafka

Re: Tuning replication

Joel, Thanks for your input - it fits what I was thinking, so it's good confirmation. The easiest mbean to look at is the underreplicated partition count. This is at the broker-level so it is coarse-grained. If it is 0 you can use various tools to do mbean queries to figure out which