[
https://issues.apache.org/jira/browse/KAFKA-714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13560156#comment-13560156
]
John Fung commented on KAFKA-714:
---------------------------------
Thanks Jun for the patch. I have verified it with the latest 0.8 branch with
250 topics & 3 partitions in a 3-broker clusters (Replication factor = 3) and
it works fine. FYI: the request-timeout-ms is set to 90,000ms (I think it is
not necessary to be this high)
> ConsoleConsumer throws SocketTimeoutException when fetching topic metadata
> --------------------------------------------------------------------------
>
> Key: KAFKA-714
> URL: https://issues.apache.org/jira/browse/KAFKA-714
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 0.8
> Reporter: John Fung
> Assignee: Jun Rao
> Priority: Blocker
> Labels: bugs, p1
> Attachments: kafka-714.patch, kafka-714-reproduce-issue.patch
>
>
> Test Description:
> 1. 1 zookeeper
> 2. 3 brokers
> 3. Replication factor = 3
> 4. Partions = 2
> 5. No. of topics = 250
> There is no problem sending messages to brokers. But ConsoleConsumer is
> throwing SocketTimeoutException when fetching topic metadata. Currently,
> ConsoleConsumer doesn't provide a command line argument to configure Socket
> timeout.
> Exception:
> [2013-01-21 10:10:08,915] WARN Fetching topic metadata with correlation id 0
> for topics [Set(topic_0219, topic_0026, topic_0160, topic_0056, topic_0100,
> topic_0146, topic_0103, topic_0179, topic_0078, topic_0098, topic_0102,
> topic_0028, topic_0060, topic_0218, topic_0210, topic_0161, topic_0144,
> topic_0101, topic_0104, topic_0186, topic_0040, topic_0027, topic_0093,
> topic_0147, topic_0080, topic_0211, topic_0089, topic_0177, topic_0220,
> topic_0097, topic_0079, topic_0187, topic_0105, topic_0178, topic_0096,
> topic_0108, topic_0095, topic_0065, topic_0066, topic_0021, topic_0023,
> topic_0109, topic_0058, topic_0092, topic_0149, topic_0150, topic_0250,
> topic_0022, topic_0227, topic_0145, topic_0063, topic_0094, topic_0216,
> topic_0185, topic_0057, topic_0141, topic_0215, topic_0184, topic_0024,
> topic_0214, topic_0140, topic_0217, topic_0228, topic_0025, topic_0064,
> topic_0044, topic_0043, topic_0152, topic_0009, topic_0029, topic_0151,
> topic_0142, topic_0041, topic_0164, topic_0077, topic_0062, topic_0163,
> topic_0046, topic_0061, topic_0190, topic_0162, topic_0143, topic_0165,
> topic_0148, topic_0042, topic_0087, topic_0223, topic_0182, topic_0008,
> topic_0132, topic_0204, topic_0007, topic_0067, topic_0181, topic_0169,
> topic_0203, topic_0180, topic_0224, topic_0183, topic_0048, topic_0107,
> topic_0069, topic_0130, topic_0106, topic_0047, topic_0068, topic_0222,
> topic_0189, topic_0221, topic_0131, topic_0134, topic_0156, topic_0111,
> topic_0246, topic_0110, topic_0245, topic_0171, topic_0240, topic_0010,
> topic_0122, topic_0201, topic_0135, topic_0196, topic_0034, topic_0241,
> topic_0012, topic_0230, topic_0082, topic_0188, topic_0195, topic_0166,
> topic_0088, topic_0036, topic_0099, topic_0172, topic_0112, topic_0085,
> topic_0202, topic_0123, topic_0011, topic_0115, topic_0084, topic_0121,
> topic_0243, topic_0086, topic_0192, topic_0035, topic_0191, topic_0200,
> topic_0242, topic_0231, topic_0133, topic_0229, topic_0116, topic_0167,
> topic_0244, topic_0032, topic_0168, topic_0157, topic_0118, topic_0209,
> topic_0045, topic_0226, topic_0119, topic_0076, topic_0117, topic_0006,
> topic_0129, topic_0225, topic_0033, topic_0159, topic_0037, topic_0197,
> topic_0030, topic_0049, topic_0205, topic_0238, topic_0004, topic_0153,
> topic_0074, topic_0127, topic_0083, topic_0003, topic_0126, topic_0249,
> topic_0158, topic_0005, topic_0081, topic_0155, topic_0031, topic_0198,
> topic_0206, topic_0020, topic_0154, topic_0075, topic_0239, topic_0128,
> topic_0212, topic_0017, topic_0054, topic_0174, topic_0073, topic_0072,
> topic_0173, topic_0039, topic_0213, topic_0138, topic_0059, topic_0015,
> topic_0055, topic_0052, topic_0237, topic_0038, topic_0091, topic_0236,
> topic_0053, topic_0234, topic_0070, topic_0193, topic_0051, topic_0090,
> topic_0248, topic_0125, topic_0002, topic_0050, topic_0247, topic_0137,
> topic_0124, topic_0014, topic_0001, topic_0071, topic_0235, topic_0194,
> topic_0120, topic_0232, topic_0175, topic_0208, topic_0170, topic_0114,
> topic_0016, topic_0139, topic_0013, topic_0136, topic_0113, topic_0018,
> topic_0233, topic_0019, topic_0176, topic_0199, topic_0207)] from broker
> [id:1,host:esv4-app19.corp.linkedin.com,port:9091] failed
> (kafka.client.ClientUtils$)
> java.net.SocketTimeoutException
> at
> sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:201)
> at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:86)
> at
> java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:221)
> at kafka.utils.Utils$.read(Utils.scala:393)
> at
> kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
> at kafka.network.Receive$class.readCompletely(Transmission.scala:56)
> at
> kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29)
> at kafka.network.BlockingChannel.receive(BlockingChannel.scala:100)
> at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
> at
> kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:71)
> at kafka.producer.SyncProducer.send(SyncProducer.scala:105)
> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:33)
> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:66)
> at
> kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.kafka$consumer$ZookeeperConsumerConnector$ZKRebalancerListener$$rebalance(ZookeeperConsumerConnector.scala:403)
> at
> kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anonfun$syncedRebalance$1.apply$mcVI$sp(ZookeeperConsumerConnector.scala:373)
> at
> scala.collection.immutable.Range$ByOne$class.foreach$mVc$sp(Range.scala:282)
> at
> scala.collection.immutable.Range$$anon$2.foreach$mVc$sp(Range.scala:265)
> at
> kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:368)
> at
> kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$$reinitializeConsumer(ZookeeperConsumerConnector.scala:674)
> at
> kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:709)
> at
> kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:141)
> at kafka.consumer.ConsoleConsumer$.main(ConsoleConsumer.scala:183)
> at kafka.consumer.ConsoleConsumer.main(ConsoleConsumer.scala)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
