Jiao Zhang created KAFKA-9372:
---------------------------------
Summary: Add producer config to make topicExpiry configurable
Key: KAFKA-9372
URL: https://issues.apache.org/jira/browse/KAFKA-9372
Project: Kafka
Issue Type: Improvement
Components: producer
Affects Versions: 1.1.0
Reporter: Jiao Zhang
Sometimes we got error "org.apache.kafka.common.errors.TimeoutException: Failed
to update metadata after 1000 ms" on producer side. We did the investigation
and found
# our producer produced messages in really low rate, the interval is more than
10 minutes
# by default, producer would expire topics after TOPIC_EXPIRY_MS, after topic
expired if no data produce before next metadata update (automatically triggered
by metadata.max.age.ms) partitions entry for the topic would disappear from the
Metadata cache As a result, almost for every time's produce, producer need
fetch metadata which could possibly end with timeout.
To solve this, we propose to add a new config metadata.topic.expiry for
producer to make topicExpiry configurable. Topic expiry is good only when
producer is long-lived and is used for producing variable counts of topics. But
in the case that producers are bounded to single or few fixed topics, there is
no need to expire topics at all.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
