[
https://issues.apache.org/jira/browse/KAFKA-608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Neha Narkhede updated KAFKA-608:
--------------------------------
Attachment: kafka-608-v2.patch
1. My bad, included in this patch
2. Absolutely, prepended "Message: ". Once this is checked in, we should update
the system tests to use it. Will file another JIRA to cover that.
3,4. Makes sense, done.
5. Thought about it and am still investigating some bugs related to topic
metadata request timeouts and large number of topics. Ultimately, in order to
allow clients to scale the # of topics they produce to/consume from, we need to
allow a much larger timeout to accomodate the get metadata request. This is
true until we move to zookeeper 3.4.x that allows us to use the multi-get APIs,
which will reduce the latency of fetching metadata from zookeeper from large
number of zk paths. However, I'm waiting for that investigation to complete and
have KAFKA-626 to cover that issue.
> getTopicMetadata does not respect producer config settings
> ----------------------------------------------------------
>
> Key: KAFKA-608
> URL: https://issues.apache.org/jira/browse/KAFKA-608
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8
> Reporter: Jay Kreps
> Assignee: Neha Narkhede
> Priority: Blocker
> Fix For: 0.8
>
> Attachments: kafka-608-v1.patch, kafka-608-v2.patch
>
>
> ProducerPool.scala contains the following code:
> object ProducerPool{
> def createSyncProducer(configOpt: Option[ProducerConfig], broker: Broker):
> SyncProducer = {
> val props = new Properties()
> props.put("host", broker.host)
> props.put("port", broker.port.toString)
> if(configOpt.isDefined)
> props.putAll(configOpt.get.props.props)
> new SyncProducer(new SyncProducerConfig(props))
> }
> }
> Note also, that ClientUtils.getTopicMetadata() does the following:
> ProducerPool.createSyncProducer(None, brokers(i))
> As a result there is no way to control the socket settings for the get
> metadata request.
> My recommendation is that we require the config to be specified in the
> Note that this creates a new sync producer without using ANY of the settings
> the user had given for the producer. In particular the socket timeout is
> defaulted to 500ms.
> This causes unit tests to fail a lot since a newly started test may easily
> timeout on a 500ms request.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
