[
https://issues.apache.org/jira/browse/STORM-297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14013473#comment-14013473
]
ASF GitHub Bot commented on STORM-297:
--------------------------------------
Github user clockfly commented on the pull request:
https://github.com/apache/incubator-storm/pull/103#issuecomment-44634492
Hi Gvain,
Firstly I apologize for the late reply, I was stuck by a 2-day training.
Thank you for sharing your data! It is really awesome that we are using the
data to present our points!
Before jumping to conclusions, let's make sure we understand the data in
the same way and the test can be reproduced, I have some questions about your
data.
1. what tps here is consist of? Is it solely consist of spout message? or
also including acker message? and other message?
2. What is the message latency in each test?
3. I noticed your network usage is pretty small, is it overall cluster
network usage? or is it only single machine network usage? Are you using 1Gb
network?
4. What is your max spout pending setting?
> Storm Performance cannot be scaled up by adding more CPU cores
> --------------------------------------------------------------
>
> Key: STORM-297
> URL: https://issues.apache.org/jira/browse/STORM-297
> Project: Apache Storm (Incubating)
> Issue Type: Bug
> Reporter: Sean Zhong
> Labels: Performance, netty
> Fix For: 0.9.2-incubating
>
> Attachments: Storm_performance_fix.pdf,
> storm_Netty_receiver_diagram.png, storm_performance_fix.patch
>
>
> We cannot scale up the performance by adding more CPU cores and increasing
> parallelism.
> For a 2 layer topology Spout ---shuffle grouping--> bolt, when message size
> is small (around 100 bytes), we can find in the below picture that neither
> the CPU nor the network is saturated. When message size is 100 bytes, only
> 40% of CPU is used, only 18% of network is used, although we have a high
> parallelism (overall we have 144 executors)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
