Get replication and partition count of a topic

2015-01-12 Thread Ankit Jain
Hi All, I want to get the replication and partition count of a topic. I tried the following piece of code: java.util.SetString topics = new HashSetString(); topics.add(topicName); SetTopicMetadata topicMetadatas =

Re: Get replication and partition count of a topic

2015-01-12 Thread Manikumar Reddy
Hi, kafka-topics.sh script can be used to retrieve topic information. Ex: sh kafka-topics.sh --zookeeper localhost:2181 --describe --topic TOPIC1 You can look into TopicCommand.scala code

Re: new to kafka

2015-01-12 Thread Otis Gospodnetic
Hi, Check http://search-hadoop.com/?q=benchmarkfc_project=Kafka for info about Kafka benchmarking. Otis -- Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr Elasticsearch Support * http://sematext.com/ On Sun, Jan 11, 2015 at 8:03 PM, Alpesh Desai desa...@gmail.com

Kafka Cluster Monitoring and Documentation of Internals (JMX Metrics) of Rejected Events

2015-01-12 Thread Bhavesh Mistry
Hi Kafka Team, I am trying to find out Kafka Internal and how a message can be corrupted or lost at brokers side. I have refer to following documentations for monitoring: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Internals http://kafka.apache.org/documentation.html#monitoring I am

Thread safety of Encoder implementations

2015-01-12 Thread Elizabeth Bennett
Hi Kafka Users, I have written my own implementation of the kafka Encoder class for serializing objects to Messages. It uses Kryo, which is a non-thread safe java serialization library. I'm using Kafka 0.7.2. We recently ran into an issue where we increased the number of kafka brokers for our

Re: Kafka Cluster Monitoring and Documentation of Internals (JMX Metrics) of Rejected Events

2015-01-12 Thread Otis Gospodnetic
Hi, I think you could just email user@? There was no attached image. I think Jun committed something about this: https://issues.apache.org/jira/browse/KAFKA-1481?focusedCommentId=14272057page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14272057 Otis -- Monitoring *

Re: Using Kafka for Event Sourcing

2015-01-12 Thread Yann Simon
up, as I am not sure you receive this email. Le Sun Jan 11 2015 at 5:34:17 PM, Yann Simon yann.simon...@gmail.com a écrit : Hi, after having read http://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying, I am considering

Re: Consumer and offset management support in 0.8.2 and 0.9

2015-01-12 Thread Joel Koshy
Catching up on email - yes the wire protocol change was a big mistake and +1 on this approach. I'm reviewing KAFKA-1841 right now. On Thu, Jan 08, 2015 at 10:07:44PM -0800, Dana Powers wrote: This approach makes sense to me -- thanks, Jun. -Dana On Thu, Jan 8, 2015 at 3:57 PM, Jun Rao

Re: Get replication and partition count of a topic

2015-01-12 Thread Ewen Cheslack-Postava
I think the closest thing to what you want is ZkUtils.getPartitionsForTopics, which returns a list of partition IDs for each topic you specify. -Ewen On Mon, Jan 12, 2015 at 12:55 AM, Manikumar Reddy ku...@nmsworks.co.in wrote: Hi, kafka-topics.sh script can be used to retrieve topic