Hi, is your kafka topic partitioned?
See: http://stackoverflow.com/questions/17205561/data-modeling-with-kafka-topics-and-partitions How is KafkaSpout configured? Regards, Tom From: trung kien [mailto:[email protected]] Sent: Mittwoch, 2. September 2015 09:05 To: [email protected] Subject: How to have multiple storm workers consume a kafka topic in parallel Hi Storm Users, I am new with Storm and using Trident for my applications. My application needs to push large of message into Kafka (in Json format), do some calculations and save the result in Redis. It seems that storm always assign only 1 worker for consuming the Kafka topic (even I have .parallelismhint(5) and my Storm cluster have 10 workers) Is there any way to have more than one worker consume a Kafka queue in parallel? Here is my topology code: topology.newStream("msg",kafkaSpout) .shuffle() .each(new Fields("str"),new JsonDecode(), new Fields("user_id","user_name")) .parallelismHint(5); Could someone please help me on this? only one worker is causing high latency in my application. -- Thanks Kien
