Hi Sai, kafkaSpout config and topic partitions looks good to me. Are you sure from your kafka producer you are writing to all the partitions in your topic. -Harsha
On Thu, Nov 13, 2014, at 07:39 PM, 张炜 wrote: > I have a question abot Kafka Spout > (https://github.com/apache/storm/tree/master/external/storm-kafka) > that need your help. > > I build Kafka Spout and set to use 7 worker to receive Kafka messages. > builder.setSpout(Utils.TOPIC_DEFAULT, new > KafkaSpout(spoutConfig_Default), 7); > > And finally I find that all messages go to only one worker, which is > shown as the diagram below. > > > > Actually my Kafka topic has multiple partitions. > > $ bin/kafka-topics.sh --topic general_dev --describe --zookeeper > localhost:2181 Topic:general_dev PartitionCount:7 ReplicationFactor:1 > Configs: Topic: general_dev Partition: 0 Leader: 1 Replicas: 1 Isr: 1 > Topic: general_dev Partition: 1 Leader: 2 Replicas: 2 Isr: 2 Topic: > general_dev Partition: 2 Leader: 3 Replicas: 3 Isr: 3 Topic: > general_dev Partition: 3 Leader: 4 Replicas: 4 Isr: 4 Topic: > general_dev Partition: 4 Leader: 5 Replicas: 5 Isr: 5 Topic: > general_dev Partition: 5 Leader: 6 Replicas: 6 Isr: 6 Topic: > general_dev Partition: 6 Leader: 7 Replicas: 7 Isr: 7 > > 1. Does the Kafka Spout support multiple worker and how to do it? > > 2. If my topic has multiple partitions, and I want each partition to > be consumed by one worker to achieve the max performance. If the > current KafkaSpout cannot do this, could you please give some > suggestions on how to make it? > > -- > Regards, Sai Email had 1 attachment: > * Screenshot.png 39k (image/png)
