@Ewen
On Fri, Mar 20, 2015 at 10:04 PM, Ewen Cheslack-Postava
wrote:
> Even if you have metadata cached, if the broker isn't available then
> messages can get stuck in the producer indefinitely. Currently the new
> producer doesn't have any client-side timeouts, which is a bug. See
> https://issu
@Mayuresh
On Fri, Mar 20, 2015 at 10:30 PM, Mayuresh Gharat
wrote:
> But if the entire kafka cluster is down, it would try for some configured
> number of retries and would return back an error in future. This is my
> understanding. Please correct me if I am wrong.
When I set the "retries" optio
I think if the leader is down, it a leader is down, the producer would
issue a metadata request and get the new leader and start producing to it.
But if the entire kafka cluster is down, it would try for some configured
number of retries and would return back an error in future. This is my
understa
Even if you have metadata cached, if the broker isn't available then
messages can get stuck in the producer indefinitely. Currently the new
producer doesn't have any client-side timeouts, which is a bug. See
https://issues.apache.org/jira/browse/KAFKA-1788 for more details.
On Fri, Mar 20, 2015 a
This is correct when you send to a topic for the first time. After that
the metadata will be cached, the metadata cache has an age and after it
expires, metadata will be refreshed.
So the time a producer found a broker is not reachable is the minimum
value of the following times:
1. Linger.ms + ret
@Sunil
The else branch will be executed if
`metadata.fetch().partitionsForTopic(topic)` returns non NULL value. I
assume that when Kafka is unreachable, it will return NULL.
`waitOnMetadata()` then returns; we never enter the else branch when
Kafka is unreachable.
@Everyone: Is this explanation c
@Samuel
My point was
The else branch of the code will be executed when metadata is not
available, and metadata is not available when kafka cluster is not rachable.
please correct me if i am wrong..
On Fri, Mar 20, 2015 at 3:43 PM, Samuel Chase wrote:
> @Sunil
>
> On Fri, Mar 20, 2015 at 3:36 PM
@Sunil
On Fri, Mar 20, 2015 at 3:36 PM, sunil kalva wrote:
> I think KafkaProducer.send method blocks until it fetches partition
> metadata for configured time using "metadata.fetch.timeout.ms", once time
> out it throws TimeoutException. You might be experiencing TimeoutException ?
My co-worker
I think KafkaProducer.send method blocks until it fetches partition
metadata for configured time using "metadata.fetch.timeout.ms", once time
out it throws TimeoutException. You might be experiencing TimeoutException ?
ref: KafkaProducer.java(waitOnMetadata)
On Fri, Mar 20, 2015 at 2:42 PM, Samue
On Fri, Mar 20, 2015 at 3:24 PM, tao xiao wrote:
> The underlining send runs in a different thread and doesn't block
> producer.send(). One way I can think of to detect this is to set
> block.on.buffer.full=false and catch BufferExhaustedException then check if
> the broker is reachable. But this
The underlining send runs in a different thread and doesn't block
producer.send(). One way I can think of to detect this is to set
block.on.buffer.full=false and catch BufferExhaustedException then check if
the broker is reachable. But this is an hacky way as BufferExhaustedException
may indicate o
@Tao,
On Fri, Mar 20, 2015 at 12:39 PM, tao xiao wrote:
> You can set producer property retries not equal to 0. Details can be found
> here
> http://kafka.apache.org/documentation.html#newproducerconfigs
I set "retries" to "1", but send is still blocking until the Kafka
Server is reachable again
Hello Tao,
On Fri, Mar 20, 2015 at 12:39 PM, tao xiao wrote:
> You can set producer property retries not equal to 0. Details can be found
> here
> http://kafka.apache.org/documentation.html#newproducerconfigs
Thanks! I shall try that.
Samuel
You can set producer property retries not equal to 0. Details can be found
here
http://kafka.apache.org/documentation.html#newproducerconfigs
On Fri, Mar 20, 2015 at 3:01 PM, Samuel Chase wrote:
> Hello Everyone,
>
> In the the new Java Producer API, the Callback code in
> KafkaProducer.send is
14 matches
Mail list logo
