Xi Wang created KAFKA-20804:
-------------------------------
Summary: Reduce lock contention in ProducerMetadata#add
Key: KAFKA-20804
URL: https://issues.apache.org/jira/browse/KAFKA-20804
Project: Kafka
Issue Type: Improvement
Components: clients
Reporter: Xi Wang
add() is called on every send() via KafkaProducer.waitOnMetadata, and the
synchronized method acquiring the instance lock causing synchronized contention
which impacts the performance.
Switch the topics map to a ConcurrentHashMap and refresh known topics
via a lock-free replace(), which only writes if the topic is still
present. For a topic seen for the first time (or one concurrently
evicted by retainTopic()), fall back to a synchronized block so the map
insert and newTopics bookkeeping happen atomically with retainTopic() -
otherwise retainTopic() could expire-and-remove the topic in the gap
between the two, leaving it recorded in newTopics without the
corresponding map entry, or silently re-inserting an evicted topic
without the newTopics bookkeeping needed to trigger an immediate
refresh.
retainTopic() switches to a conditional remove(topic, expireMs) so a
concurrent refresh of an existing topic in add() can't be undone by an
eviction decision based on the expiry value read just before the refresh
landed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
