[jira] [Updated] (KAFKA-4385) producer is sending too many unnecessary meta data request if the meta data for a topic is not available and "auto.create.topics.enable" =false
[ https://issues.apache.org/jira/browse/KAFKA-4385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stevo Slavic updated KAFKA-4385: Affects Version/s: 0.9.0.1 > producer is sending too many unnecessary meta data request if the meta data > for a topic is not available and "auto.create.topics.enable" =false > --- > > Key: KAFKA-4385 > URL: https://issues.apache.org/jira/browse/KAFKA-4385 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.9.0.1 >Reporter: Jun Yao > > All current kafka-client producer implementation (<= 0.10.1.0), > When sending a msg to a topic, it will first check if meta data for this > topic is available or not, > when not available, it will set "metadata.requestUpdate()" and wait for meta > data from brokers, > The thing is inside "org.apache.kafka.clients.Metadata.awaitUpdate()", it's > already doing a "while (this.version <= lastVersion)" loop waiting for new > version response, > So the loop inside > "org.apache.kafka.clients.producer.KafkaProducer.waitOnMetadata() is not > needed, > When "auto.create.topics.enable" is false, sending msgs to a non-exist topic > will trigger too many meta requests, everytime a metadata response is > returned, because it does not contain the metadata for the topic, it's going > to try again until TimeoutException is thrown; > This is a waste and sometimes causes too much overhead when unexpected msgs > are arrived. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-4385) producer is sending too many unnecessary meta data request if the meta data for a topic is not available and "auto.create.topics.enable" =false
[ https://issues.apache.org/jira/browse/KAFKA-4385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Yao updated KAFKA-4385: --- Summary: producer is sending too many unnecessary meta data request if the meta data for a topic is not available and "auto.create.topics.enable" =false (was: producer is sending too many unnecessary meta data request if the meta data for a topic is not available) > producer is sending too many unnecessary meta data request if the meta data > for a topic is not available and "auto.create.topics.enable" =false > --- > > Key: KAFKA-4385 > URL: https://issues.apache.org/jira/browse/KAFKA-4385 > Project: Kafka > Issue Type: Bug >Reporter: Jun Yao > > All current kafka-client producer implementation (<= 0.10.1.0), > When sending a msg to a topic, it will first check if meta data for this > topic is available or not, > when not available, it will set "metadata.requestUpdate()" and wait for meta > data from brokers, > The thing is inside "org.apache.kafka.clients.Metadata.awaitUpdate()", it's > already doing a "while (this.version <= lastVersion)" loop waiting for new > version response, > So the loop inside > "org.apache.kafka.clients.producer.KafkaProducer.waitOnMetadata() is not > needed, > When "auto.create.topics.enable" is false, sending msgs to a non-exist topic > will trigger too many meta requests, everytime a metadata response is > returned, because it does not contain the metadata for the topic, it's going > to try again until TimeoutException is thrown; > This is a waste and sometimes causes too much overhead when unexpected msgs > are arrived. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
