Could you try the latest code in the 0.8 branch? We recently fixed a consumer side issue (KAFKA-1030) that seems related.
Thanks, Jun On Thu, Oct 10, 2013 at 6:36 AM, Viktor Kolodrevskiy < viktor.kolodrevs...@gmail.com> wrote: > On our current project we want to have a single shell script that: > first starts kafka with zookeeper and second strarts tomcat server > with our webapp. > We run kafka and tomcat on single machine, use kafka 0.8 > We set num.partitions=4 in server.properties. > In webapp we start 4 threads that do > kafka.consumer.Consumer.createJavaConsumerConnector to create consumer > connectors. > > The logic of script is following: > 1. run zookeeper-server-start.sh and wait until zk process > appears(with linux pgrep command). > 2. run kafka-server-start.sh and wait until kafka process appears. > 3. sleep a fixed timeout(say 10 seconds) to let kafka completely initialize > 4. start tomcat > > If timeout in step#3 is smaller we get following warning and consumers > do not receive messages: > WARN [kafka.consumer.ZookeeperConsumerConnector] > [test-1381405301679-580f9706], No broker partitions consumed by > consumer thread test-1381405301679-580f9706-3 for topic test-topic > > Is there a way to avoid this hardcoded timeout? How to check that > partitions are initialized? > > > Thanks, > Viktor >
