Re: Help on understanding kafka-topics.sh output

2015-11-22 Thread Jan Algermissen
Hi Todd, thanks very much for the explanations here and in the previous mail - helps a lot! (I assumed the callback would provide an exception when a message could not be sent - I observe the callback being called for every message to the up broker, but yes, maybe the developers can clarify how

Re: Help on understanding kafka-topics.sh output

2015-11-22 Thread Todd Palino
Hopefully one of the developers can jump in here. I believe there is a future you can use to get the errors back from the producer. In addition, you should check the following configs on the producer: request.required.acks - this controls whether or not your producer is going to wait for an acknow

Re: Help on understanding kafka-topics.sh output

2015-11-22 Thread Jan Algermissen
Hi Todd, yes, correct - thanks. However, what I am not getting is that the KafkaProducer (see my other mail from today) silently accepts the messages and fills them up in the buffer until it is exhausted instead of saying that the broker is not reachable. IOW, it seems from an application pers

Re: Help on understanding kafka-topics.sh output

2015-11-22 Thread Todd Palino
Replicas and Isr are both a comma separated list of broker IDs. So in this output, I am seeing that you have two Kafka brokers with IDs 1 and 2. You have a topic, capture, with 16 partitions at replication factor 1 (1 replica per partition). The broker with ID 2 is not online, which is why it shows

Help on understanding kafka-topics.sh output

2015-11-22 Thread Jan Algermissen
Hi, I have a topic with 16 partitions that shows the following output for kafka-topics.sh --zookeeper x:2181 --topic capture --describe Can anyone explain to me what the difference in replicas means and what Leader of -1 means? In the logs of my producer I see that no messages seem to be