This worked. Thank you.
> On Jun 25, 2014, at 7:04 AM, Joe Stein <joe.st...@stealth.ly> wrote: > > Zach, that error will occur if different brokers are advertising themselves > in such a way where they are resolving to the same IP address. The > advertised hostname is the hostname that will be given out to producers, > consumers, and other brokers to connect to from the fetch meta data request. > > Check what you have set as the advertised hostname value in > server.properties for both of your brokers... If that is not set then > something is wrong with your server/network configurations and DNS is > resolving each broker's hostname to the same IP as a result of each box > having to default to java.net.InetAddress.getCanonicalHostName() without > that value set. > > /******************************************* > Joe Stein > Founder, Principal Consultant > Big Data Open Source Security LLC > http://www.stealth.ly > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> > ********************************************/ > > >> On Tue, Jun 24, 2014 at 11:24 PM, Zack Payton <zpay...@gmail.com> wrote: >> >> server.log has a lot of these errors: >> [2014-06-24 20:07:16,124] ERROR [KafkaApi-6] error when handling request >> Name: FetchRequest; Version: 0; CorrelationId: 81138; ClientId: >> ReplicaFetche >> rThread-0-5; ReplicaId: 6; MaxWait: 500 ms; MinBytes: 1 bytes; RequestInfo: >> [test1,0] -> PartitionFetchInfo(0,1048576) (kafka.server.KafkaApis) >> *kafka.common.KafkaException: Shouldn't set logEndOffset for replica 6 >> partition [test1,0] since it's local* >> at kafka.cluster.Replica.logEndOffset_$eq(Replica.scala:46) >> at >> >> kafka.cluster.Partition.updateLeaderHWAndMaybeExpandIsr(Partition.scala:236) >> at >> >> kafka.server.ReplicaManager.recordFollowerPosition(ReplicaManager.scala:432) >> at >> >> kafka.server.KafkaApis$$anonfun$maybeUpdatePartitionHw$2.apply(KafkaApis.scala:460) >> at >> >> kafka.server.KafkaApis$$anonfun$maybeUpdatePartitionHw$2.apply(KafkaApis.scala:458) >> at scala.collection.immutable.Map$Map1.foreach(Map.scala:105) >> at >> kafka.server.KafkaApis.maybeUpdatePartitionHw(KafkaApis.scala:458) >> at kafka.server.KafkaApis.handleFetchRequest(KafkaApis.scala:424) >> at kafka.server.KafkaApis.handle(KafkaApis.scala:186) >> at >> kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:42) >> at java.lang.Thread.run(Thread.java:701) >> >> >> >>> On Tue, Jun 24, 2014 at 8:10 PM, Joe Stein <joe.st...@stealth.ly> wrote: >>> >>> Are there any errors in the broker's logs? >>> >>> /******************************************* >>> Joe Stein >>> Founder, Principal Consultant >>> Big Data Open Source Security LLC >>> http://www.stealth.ly >>> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> >>> ********************************************/ >>> >>> >>>> On Tue, Jun 24, 2014 at 9:56 PM, Zack Payton <zpay...@gmail.com> wrote: >>>> >>>> Hi all, >>>> >>>> I have 3 zookeeper servers and 2 Kafka servers. >>>> Running Kafka version 0.8.1.1. >>>> Running zookeeper 3.3.5-cdh3u6. >>>> From the Kafka servers I can access the zookeeper servers on 2181. >>>> From one of the Kafka servers I can create a topic no problem: >>>> [root@kafka1 kafka-0.8.1.1-src]# bin/kafka-topics.sh --zookeeper >>>> zookeeper1:2181,zookeeper2:2181,zookeeper3 --create --topic test1 >>>> --replication-factor 2 --partitions 2 >>>> Created topic "test1". >>>> >>>> But, when I go to write to it I get the following: >>>> [root@kafka1 kafka-0.8.1.1-src]# bin/kafka-console-producer.sh >>>> --broker-list kafka1.queue.customer.sea1.ops.riotgames.com:9092, >>>> kafka2.queue.customer.sea1.ops.riotgames.com:9092 --topic test1 >>>> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". >>>> SLF4J: Defaulting to no-operation (NOP) logger implementation >>>> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for >>> further >>>> details. >>>> TEST1 >>>> [2014-06-24 18:49:15,753] WARN Error while fetching metadata >>>> [{TopicMetadata for topic test1 -> >>>> No partition metadata for topic test1 due to >>>> kafka.common.LeaderNotAvailableException}] for topic [test1]: class >>>> kafka.common.LeaderNotAvailableException >>>> (kafka.producer.BrokerPartitionInfo) >>>> [2014-06-24 18:49:15,813] WARN Error while fetching metadata partition >>>> 0 leader: >>>> none replicas: isr: isUnderReplicated: false for topic partition >>> [test1,0]: >>>> [class kafka.common.LeaderNotAvailableException] >>>> (kafka.producer.BrokerPartitionInfo) >>>> [2014-06-24 18:49:15,814] WARN Error while fetching metadata partition >>>> 1 leader: >>>> none replicas: isr: isUnderReplicated: false for topic partition >>> [test1,1]: >>>> [class kafka.common.LeaderNotAvailableException] >>>> (kafka.producer.BrokerPartitionInfo) >>>> [2014-06-24 18:49:15,821] WARN Failed to collate messages by >>>> topic,partition due to: No leader for any partition in topic test1 >>>> (kafka.producer.async.DefaultEventHandler) >>>> [2014-06-24 18:49:15,933] WARN Error while fetching metadata >>>> [{TopicMetadata for topic test1 -> >>>> No partition metadata for topic test1 due to >>>> kafka.common.LeaderNotAvailableException}] for topic [test1]: class >>>> kafka.common.LeaderNotAvailableException >>>> (kafka.producer.BrokerPartitionInfo) >>>> [2014-06-24 18:49:15,933] WARN Failed to collate messages by >>>> topic,partition due to: No leader for any partition in topic test1 >>>> (kafka.producer.async.DefaultEventHandler) >>>> [2014-06-24 18:49:16,041] WARN Error while fetching metadata >>>> [{TopicMetadata for topic test1 -> >>>> No partition metadata for topic test1 due to >>>> kafka.common.LeaderNotAvailableException}] for topic [test1]: class >>>> kafka.common.LeaderNotAvailableException >>>> (kafka.producer.BrokerPartitionInfo) >>>> [2014-06-24 18:49:16,042] WARN Failed to collate messages by >>>> topic,partition due to: No leader for any partition in topic test1 >>>> (kafka.producer.async.DefaultEventHandler) >>>> [2014-06-24 18:49:16,148] WARN Error while fetching metadata partition >>>> 0 leader: >>>> none replicas: isr: isUnderReplicated: false for topic partition >>> [test1,0]: >>>> [class kafka.common.LeaderNotAvailableException] >>>> (kafka.producer.BrokerPartitionInfo) >>>> [2014-06-24 18:49:16,149] WARN Error while fetching metadata partition >>>> 1 leader: >>>> none replicas: isr: isUnderReplicated: false for topic partition >>> [test1,1]: >>>> [class kafka.common.LeaderNotAvailableException] >>>> (kafka.producer.BrokerPartitionInfo) >>>> [2014-06-24 18:49:16,149] WARN Failed to collate messages by >>>> topic,partition due to: No leader for any partition in topic test1 >>>> (kafka.producer.async.DefaultEventHandler) >>>> [2014-06-24 18:49:16,257] WARN Error while fetching metadata >>>> [{TopicMetadata for topic test1 -> >>>> No partition metadata for topic test1 due to >>>> kafka.common.LeaderNotAvailableException}] for topic [test1]: class >>>> kafka.common.LeaderNotAvailableException >>>> (kafka.producer.BrokerPartitionInfo) >>>> [2014-06-24 18:49:16,260] ERROR Failed to send requests for topics >> test1 >>>> with correlation ids in [0,8] >> (kafka.producer.async.DefaultEventHandler) >>>> [2014-06-24 18:49:16,262] ERROR Error in handling batch of 1 events >>>> (kafka.producer.async.ProducerSendThread) >>>> kafka.common.FailedToSendMessageException: Failed to send messages >> after >>> 3 >>>> tries. >>>> at >> kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90) >>>> at >> kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:104) >>>> at >> kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:87) >>>> at >> kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:67) >>>> at scala.collection.immutable.Stream.foreach(Stream.scala:254) >>>> at >> kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:66) >>>> at >>> kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:44) >>>> >>>> >>>> >>>> Documentation on this issue is sparse. Any ideas? >>>> >>>> Thanks, >>>> Z >>
