[
https://issues.apache.org/jira/browse/STORM-814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14541117#comment-14541117
]
Sriharsha Chintalapani commented on STORM-814:
----------------------------------------------
[~rajeshkuch] This is expected behavior. If you want to get optimal throughput
your partitions and spout parallelism should match. In your case when you have
16 parttioins and 1 spout parallelism all of the topic partitions will be
assigned 1 spout to read from and when you increased it to 8 each of the spout
instance gets 2 per spout instance.
> Kafka Spout performance
> -----------------------
>
> Key: STORM-814
> URL: https://issues.apache.org/jira/browse/STORM-814
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-kafka
> Affects Versions: 0.9.3
> Environment: Centos, AWS , M3.2xlarge instances
> Reporter: Rajesh Kucharlapati
>
> I am running few test for storm topology with kafka.
> Created a topic with 16 partitions and emitted 10k messages/sec with each
> message 1k size.
> When I set the kafka spout parallelism to 1, I am getting latency of 8.9 ms
> but when i increase the parallelism to 8, I am getting latency of 1.2 sec.
> Settings :
> MAX_SPOUT SPENDING set to 1000-5000
> String zkConnString = "zookeeper1:2181";
> String kafkaTopic = "messages";
> String zkRootPath = "/kafkaStorm";
> String zkOffSetID = "kafka";
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
