[
https://issues.apache.org/jira/browse/KAFKA-1494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14154518#comment-14154518
]
Josh Rosen commented on KAFKA-1494:
-----------------------------------
I'm able to reproduce a similar problem using Kafka 0.8 with a single broker
running inside of Docker (using boot2docker) on OS X. In this setup, I run a
single-node ZooKeeper process in its own Docker container and configure the
Kafka broker to use it. The broker is configured to bind to the Docker
container's IP on the VirtualBox network. Using {{sudo route \-n add
172.17.0.0/16 `boot2docker ip`}}, I've configured my Mac so that programs
running outside of Docker can access network services running inside of
containers using the containers' own IP addresses (so we use the same IPs for
host-container and container-container communication). I have some Scala code
which uses Kafka's ZkUtils to retrieve information on the Kafka cluster state
and kafka.producer.Producer to push data into Kafka.
When I attempt to run this, I see errors very similar to the ones reported in
this ticket:
{code}
14/10/01 00:22:54 INFO client.ClientUtils$: Fetching metadata from broker
id:0,host:172.17.0.187,port:12345 with correlation id 11 for 1 topic(s)
Set(test-topic)
14/10/01 00:22:54 INFO producer.SyncProducer: Connected to 172.17.0.187:12345
for producing
14/10/01 00:22:54 INFO producer.SyncProducer: Disconnecting from
172.17.0.187:12345
14/10/01 00:22:54 WARN producer.BrokerPartitionInfo: Error while fetching
metadata [{TopicMetadata for topic test-topic ->
No partition metadata for topic test-topic due to
kafka.common.LeaderNotAvailableException}] for topic [test-topic]: class
kafka.common.LeaderNotAvailableException
{code}
I've found messages on the mailing list that describe this
LeaderNotAvailableException; however, I don't think that any of the root causes
reported there apply in my case:
- My producer can connect to the broker (I know this because I set `client.id`
in my producer configuration and see matching entries in the broker's logs).
- My broker can connect to ZooKeeper (I manually inspected ZooKeeper's contents
using zookeeper-shell).
- I didn't attempt to delete any topics.
- ZooKeeper starts with a completely fresh state; leftover state from earlier
attempts / experiments isn't an issue here.
- I don't think there are any race-conditions in my client code; I added long
(20-30 second) Thread.sleep() calls in my program so that we wait for leader
election / metadata propagation to occur after creating the new topic.
I've noticed that ZkUtils.getReplicasForPartition lists the expected replica
for the partition, although ZkUtils.getLeaderForPartition returns None, even
after multiple attempts / long delays.
This test is easy for me to re-run; the entire setup / teardown of the
containers takes less than a minute. Therefore, it's very easy for me to
experiment with different configurations / settings. I can provide any logs,
configuration files, etc. that would be helpful when debugging this; just let
me know what you need.
> Failed to send messages after 3 tries.
> --------------------------------------
>
> Key: KAFKA-1494
> URL: https://issues.apache.org/jira/browse/KAFKA-1494
> Project: Kafka
> Issue Type: Bug
> Components: controller, core
> Affects Versions: 0.8.1.1
> Environment: Mac OS
> Reporter: darion yaphets
> Assignee: Neha Narkhede
>
> I use default server & zookeeper config to start-up zookeeper server and
> kafka broker on my machine to test custom message which based on proto buffer
> . I write a client to send protobuf-message to kafka broker and source code
> as following :
> Properties properties = new Properties();
> properties.put("serializer.class",
> "java_example.ProtoBufMessage");
> properties.put("metadata.broker.list", "localhost:9092");
> ProducerConfig config = new ProducerConfig(properties);
> testBuf buffer = testBuf.newBuilder().setID(0)
> .setUrl("darion.yaphet.org").build();
> Producer<String, testBuf> producer = new Producer<String,
> testBuf>(
> config);
> producer.send(new KeyedMessage<String, testBuf>("protobuffer",
> buffer));
> client debug log report a exception:
> [FileSystemMoniter] INFO [main] kafka.utils.Logging$class.info(68) |
> Disconnecting from localhost:9092
> [FileSystemMoniter] DEBUG [main] kafka.utils.Logging$class.debug(52) |
> Successfully fetched metadata for 1 topic(s) Set(protobuffer)
> [FileSystemMoniter] WARN [main] kafka.utils.Logging$class.warn(83) | Error
> while fetching metadata [{TopicMetadata for topic protobuffer ->
> No partition metadata for topic protobuffer due to
> kafka.common.LeaderNotAvailableException}] for topic [protobuffer]: class
> kafka.common.LeaderNotAvailableException
> [FileSystemMoniter] ERROR [main] kafka.utils.Logging$class.error(97) | Failed
> to send requests for topics protobuffer with correlation ids in [0,8]
> Exception in thread "main" kafka.common.FailedToSendMessageException: Failed
> to send messages after 3 tries.
> at
> kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)
> at kafka.producer.Producer.send(Producer.scala:76)
> at kafka.javaapi.producer.Producer.send(Producer.scala:33)
> at java_example.ProducerExamples.main(ProducerExamples.java:26)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
