Hi Igor, It looks from the stack trace that the consumer is having trouble finding the leader for topic "Test1". Can you execute on one of your brokers, perhaps the one with ip address 10.1.132.113 the following command:
$KAFKA_HOME/bin/kafka-topics.sh --describe --zookeeper <urls> --topic Test1 and let me know what the output is. Also another test that you can do to take NiFi out of the picture is to use the console consumer. You can execute that with the following command, again from one of your Kafka brokers. $KAFKA_HOME/bin/kafka-console-consumer.sh --zookeeper <urls> --topic Test1 If you do not have a process writing any data into kafka, then you can add the following parameter to the above command "--from-beginning" Looking forward to hearing back from you. Thanks, Andrew On Thu, May 26, 2016 at 5:44 PM, Igor Kravzov <[email protected]> wrote: > Thanks Oleg. I'll try. If you find more data please let me know.. > > On Thu, May 26, 2016 at 5:39 PM, Oleg Zhurakousky < > [email protected]> wrote: > >> Igor, you may want to go they Kafka mailing list. I've seen this several >> times, and all I know it has to do with broker >> I'll look in more details once back on line >> >> Oleg >> >> Sent from my iPhone >> >> On May 26, 2016, at 17:20, Igor Kravzov <[email protected]> wrote: >> >> Why I am getting exception bellow >> >> >> 2016-05-26 17:13:36,776 INFO >> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] >> kafka.consumer.ConsumerFetcherManager >> [ConsumerFetcherManager-1464297163222] Added fetcher for partitions >> ArrayBuffer() >> 2016-05-26 17:13:36,981 INFO >> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] >> kafka.utils.VerifiableProperties Verifying properties >> 2016-05-26 17:13:36,981 INFO >> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] >> kafka.utils.VerifiableProperties Property client.id is overridden to >> NiFi-d217bceb-d58b-4031-9143-26d8ddd15e40 >> 2016-05-26 17:13:36,981 INFO >> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] >> kafka.utils.VerifiableProperties Property metadata.broker.list is >> overridden to 10.1.132.113:6667 >> 2016-05-26 17:13:36,981 INFO >> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] >> kafka.utils.VerifiableProperties Property request.timeout.ms is >> overridden to 30000 >> 2016-05-26 17:13:36,981 INFO >> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] >> kafka.client.ClientUtils$ Fetching metadata from broker >> id:1001,host:10.1.132.113,port:6667 with correlation id 256 for 1 topic(s) >> Set(Test1) >> 2016-05-26 17:13:36,981 INFO >> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] >> kafka.producer.SyncProducer Connected to 10.1.132.113:6667 for producing >> 2016-05-26 17:13:36,981 INFO >> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] >> kafka.producer.SyncProducer Disconnecting from 10.1.132.113:6667 >> 2016-05-26 17:13:36,981 WARN >> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] >> kafka.client.ClientUtils$ Fetching topic metadata with correlation id 256 >> for topics [Set(Test1)] from broker [id:1001,host:10.1.132.113,port:6667] >> failed >> java.nio.channels.ClosedChannelException: null >> at kafka.network.BlockingChannel.send(BlockingChannel.scala:100) >> ~[kafka_2.10-0.8.2.2.jar:na] >> at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73) >> ~[kafka_2.10-0.8.2.2.jar:na] >> at >> kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72) >> ~[kafka_2.10-0.8.2.2.jar:na] >> at kafka.producer.SyncProducer.send(SyncProducer.scala:113) >> ~[kafka_2.10-0.8.2.2.jar:na] >> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58) >> [kafka_2.10-0.8.2.2.jar:na] >> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:93) >> [kafka_2.10-0.8.2.2.jar:na] >> at >> kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66) >> [kafka_2.10-0.8.2.2.jar:na] >> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60) >> [kafka_2.10-0.8.2.2.jar:na] >> 2016-05-26 17:13:36,981 INFO >> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] >> kafka.producer.SyncProducer Disconnecting from 10.1.132.113:6667 >> 2016-05-26 17:13:36,981 WARN >> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] >> k.c.ConsumerFetcherManager$LeaderFinderThread >> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread], >> Failed to find leader for Set([Test1,0]) >> kafka.common.KafkaException: fetching topic metadata for topics >> [Set(Test1)] from broker [ArrayBuffer(id:1001,host:10.1.132.113,port:6667)] >> failed >> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:72) >> ~[kafka_2.10-0.8.2.2.jar:na] >> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:93) >> ~[kafka_2.10-0.8.2.2.jar:na] >> at >> kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66) >> ~[kafka_2.10-0.8.2.2.jar:na] >> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60) >> [kafka_2.10-0.8.2.2.jar:na] >> Caused by: java.nio.channels.ClosedChannelException: null >> at kafka.network.BlockingChannel.send(BlockingChannel.scala:100) >> ~[kafka_2.10-0.8.2.2.jar:na] >> at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73) >> ~[kafka_2.10-0.8.2.2.jar:na] >> at >> kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72) >> ~[kafka_2.10-0.8.2.2.jar:na] >> at kafka.producer.SyncProducer.send(SyncProducer.scala:113) >> ~[kafka_2.10-0.8.2.2.jar:na] >> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58) >> ~[kafka_2.10-0.8.2.2.jar:na] >> ... 3 common frames omitted >> >> > -- Thanks, Andrew Subscribe to my book: Streaming Data <http://manning.com/psaltis> <https://www.linkedin.com/pub/andrew-psaltis/1/17b/306> twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>
