[
https://issues.apache.org/jira/browse/KAFKA-788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Neha Narkhede reassigned KAFKA-788:
-----------------------------------
Assignee: Neha Narkhede (was: Jun Rao)
> Periodic refresh of topic metadata on the producer doesn't include all topics
> -----------------------------------------------------------------------------
>
> Key: KAFKA-788
> URL: https://issues.apache.org/jira/browse/KAFKA-788
> Project: Kafka
> Issue Type: Bug
> Components: producer
> Affects Versions: 0.8
> Reporter: Neha Narkhede
> Assignee: Neha Narkhede
> Labels: kafka-0.8, p2
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> We added a patch to the producer to refresh the metadata for all topics
> periodically. However, the producer only does this for the topics in the last
> batch. But some topics sent by the producer could be low throughput and might
> not be present in every batch. If we bounce the cluster or if brokers fail
> and leaders change, the metadata for those low throughput topic is not
> refreshed by this periodic topic metadata request. The next produce request
> for those topics have to fail and then a separate metadata request needs to
> be reissued to handle the produce request. This is especially a problem for
> the migration tool. So even if the producer had a chance to refresh the
> metadata when the leader changed, it throws LeaderNotAvailableExceptions much
> later when it sends a request for that topic.
> I propose we just fetch data for all topics sent by the producer in the
> periodic refresh of topic metadata
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
