leader:none question

2013-10-28 Thread linghongbo008
Hi All I encountered an error. I delete a certain topic and recreate it , but when I run the shell kafka-list-topic.sh , the topic of all partitions leader:none. I have no idea! Please tell me what happened. Thanks. topic: mqttmsglog partition: 0leader: nonereplicas:

Re: leader:none question

2013-10-28 Thread Neha Narkhede
There is no way to delete topics in Kafka yet. So trying to delete topics like this might cause failures and cause the topics to be in a bad state. Thanks, Neha On Oct 28, 2013 4:46 AM, linghongbo008 linghongbo...@gmail.com wrote: Hi All I encountered an error. I delete a certain topic and

Re: How to run a single unit test with ./sbt

2013-10-28 Thread Roger Hoover
Joel, Thank you! This is very helpful. What I notice now is that it works for Test classes that extend org.scalatest.junit.JUnit3Suite. There are other tests in the codebase that use a @Test annotation as in the example below. Any idea how to run those? import org.junit.{Test, After, Before}

Re: Limitation on kafka topic size

2013-10-28 Thread Suchi Amalapurapu
We are using kafka 0.7.2 Suchi On Tue, Oct 29, 2013 at 12:53 AM, Suchi Amalapurapu su...@bloomreach.comwrote: All Are there any limitations on max topic size in Kafka. How large can a topic get? What are the implications on resources, latencies, performance and feasability? What are the

Limitation on kafka topic size

2013-10-28 Thread Suchi Amalapurapu
All Are there any limitations on max topic size in Kafka. How large can a topic get? What are the implications on resources, latencies, performance and feasability? What are the pros and cons of using a single large topic vs multiple smaller topics? Suchi

Re: Broker bind address versus published hostname in ZooKeeper

2013-10-28 Thread Roger Hoover
OK. I've submitted a patch along with unit tests. This change is backward compatible with two new optional parameters called advertise.host.name and advertise.port. The reason I called them that instead of zkHost.name and zkHost.port is that the zkHost.* names sound like they represent the host

Cannot start Kafka 0.8, exception in leader election

2013-10-28 Thread Nicholas Tietz
Hi everyone, I'm having some problems getting Kafka 0.8 running. (I'm running this on Mac OS X 10.8 with the latest updates; Java version 1.6.0_51.) I followed the instructions from the quickstart for 0.8, both from-source and from-binary (cleaned up all related files each time). However, I am

Re: Limitation on kafka topic size

2013-10-28 Thread Guozhang Wang
Hello Suchi, Each topic will have its own partitions and each partition is represented as a list of file segments. Hence having multiple smaller topics means you will have # topics * avg. # partitions_per_topic file handlers. Volume is usually not a problem for Kafka topics, however, if you have

Re: producer exceptions when broker dies

2013-10-28 Thread Guozhang Wang
I think you are right. I was too quick saying that pre 0.7 we also have at-least-once. Guozhang On Sun, Oct 27, 2013 at 9:51 AM, Jason Rosenberg j...@squareup.com wrote: Perhaps, it's a matter of semantics. But, I think I'm not talking only about failure, but normal operation. It's normal

Recovering High Level Consumer from exceptions

2013-10-28 Thread Joseph Lawson
Hi everyone, how should a high level consumer implementation handle consumer exceptions. Using the high level consumer wiki example, the primary thread calls an example.run() and then waits for a thread.sleep and then shuts down. If each ConsumerTest is a thread, what is the best way for the

Re: Re: leader:none question

2013-10-28 Thread linghongbo008
Hi, Neha ! I use kafka0.8 that offered to delete the function of the topic. It can delete the topic by running the shell script bin/kafka-delete-topic.sh and create the topic by running the script kafka-create-topic.sh . Now, how do I fix this error to the topic of leader isn't leader :none

Re: Recovering High Level Consumer from exceptions

2013-10-28 Thread Jun Rao
Normally, consumers don't timeout unless you have configured it. You just need to make sure you don't kill the consumer thread if it hits an application error. Thanks, Jun On Mon, Oct 28, 2013 at 5:21 PM, Joseph Lawson jlaw...@roomkey.com wrote: Hi everyone, how should a high level