Thanks Jiangjie. This is helpful.
Adding to what you have mentioned, I can think of one more scenario which
may not be very rare.
Say, the application is rebooted and the Kafka brokers registered in the
producer are not reachable ( could be due to network issues or those
brokers are actually down
Isn’t the producer part of the application? The metadata is stored in
memory. If the application rebooted (process restarted), all the metadata
will be gone.
Jiangjie (Becket) Qin
On 5/13/15, 9:54 AM, Mohit Gupta success.mohit.gu...@gmail.com wrote:
I meant the producer. ( i.e. application
By application rebooting, do you mean you bounce the brokers?
Thanks,
Mayuresh
On Wed, May 13, 2015 at 4:06 AM, Mohit Gupta success.mohit.gu...@gmail.com
wrote:
Thanks Jiangjie. This is helpful.
Adding to what you have mentioned, I can think of one more scenario which
may not be very rare.
I meant the producer. ( i.e. application using the producer api to push
messages into kafka ) .
On Wed, May 13, 2015 at 10:20 PM, Mayuresh Gharat
gharatmayures...@gmail.com wrote:
By application rebooting, do you mean you bounce the brokers?
Thanks,
Mayuresh
On Wed, May 13, 2015 at 4:06
The way it works I suppose is that, the producer will do fetchMetadata, if
the last fetched metadata is stale (the refresh interval has expired) or if
it is not able to send data to a particular broker in its current metadata
(This might happen in some cases like if the leader moves).
It cannot
I could not follow the reasoning behind blocking the send method if the
metadata is not up-to-date. Though, I see that it as per design, it
requires the metadata to batch the message into appropriate topicPartition
queue. Also, if the metadata could not be updated in the specified
interval, it
That¹s right. Send() will first try to get metadata of a topic, that is a
blocking operation.
On 5/12/15, 2:48 AM, Rendy Bambang Junior rendy.b.jun...@gmail.com
wrote:
Hi, sorry if my understanding is incorrect.
I am integrating kafka producer with application, when i try to shutdown
all kafka
Send() will only block if the metadata is *not available* for the topic.
It won’t block if metadata there is stale. The metadata refresh is async
to send(). However, if you send the message to a topic for the first time,
send() will trigger a metadata refresh and block until it has metadata for
