[
https://issues.apache.org/jira/browse/KAFKA-4385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15642789#comment-15642789
]
Jiangjie Qin commented on KAFKA-4385:
-------------------------------------
It seems you are more worrying about the blocking time, right? I am curious why
it is important to optimize this specific case to fail faster. Can you tolerate
long blocking time in other cases? We used to have many fine grained
configurations and we worked hard to move away from that because it becomes
harder and harder to manage and use. The idea behind only having one
configuration of {{max.block.ms}} is that {{producer.send()}} can block at most
that long in the worst case. It is like an SLA and much intuitive to the users.
So it may be more important to understand what is the actual requirement of
your use case here before we try to optimize for this specific case.
For this specific case, it sounds like the producer should not send a message
to a non-existing topic to begin with if auto creation is off on purpose.
Besides, it is a little weird for the producer to presume the auto topic
creation is off on the broker side.
Yeah, reusing retry may be a little risky here.
> producer is sending too many unnecessary meta data request if the meta data
> for a topic is not available and "auto.create.topics.enable" =false
> -----------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-4385
> URL: https://issues.apache.org/jira/browse/KAFKA-4385
> Project: Kafka
> Issue Type: Bug
> Reporter: Jun Yao
>
> All current kafka-client producer implementation (<= 0.10.1.0),
> When sending a msg to a topic, it will first check if meta data for this
> topic is available or not,
> when not available, it will set "metadata.requestUpdate()" and wait for meta
> data from brokers,
> The thing is inside "org.apache.kafka.clients.Metadata.awaitUpdate()", it's
> already doing a "while (this.version <= lastVersion)" loop waiting for new
> version response,
> So the loop inside
> "org.apache.kafka.clients.producer.KafkaProducer.waitOnMetadata() is not
> needed,
> When "auto.create.topics.enable" is false, sending msgs to a non-exist topic
> will trigger too many meta requests, everytime a metadata response is
> returned, because it does not contain the metadata for the topic, it's going
> to try again until TimeoutException is thrown;
> This is a waste and sometimes causes too much overhead when unexpected msgs
> are arrived.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
