[
https://issues.apache.org/jira/browse/STORM-297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14017459#comment-14017459
]
ASF GitHub Bot commented on STORM-297:
--------------------------------------
Github user miguno commented on the pull request:
https://github.com/apache/incubator-storm/pull/103#issuecomment-45060188
@clockfly Beyond the specific work on this item (STORM-297) what would be
your lessons learned after having investigated Storm's performance and
scalability so closely, i.e. with regards to potential bottlenecks (= things
that are broken and that need fixing) or areas where we still have untapped
potential (= where we could optimize further, maybe simply because we haven't
focused on a particular piece of the architecture/code/... yet)?
For instance, would you say that there are some general design-related
decisions that could put an upper bound on Storm's scalability (in theory
and/or in practice)? Or would you say it is, at least at this point in time,
more the current implementation of the design? Just thinking aloud.
I'd appreciate any comments -- positive or negative -- you might have in
this context.
> Storm Performance cannot be scaled up by adding more CPU cores
> --------------------------------------------------------------
>
> Key: STORM-297
> URL: https://issues.apache.org/jira/browse/STORM-297
> Project: Apache Storm (Incubating)
> Issue Type: Bug
> Reporter: Sean Zhong
> Labels: Performance, netty
> Fix For: 0.9.2-incubating
>
> Attachments: Storm_performance_fix.pdf,
> storm_Netty_receiver_diagram.png, storm_conf.txt,
> storm_performance_fix.patch, worker_throughput_without_storm-297.png
>
>
> We cannot scale up the performance by adding more CPU cores and increasing
> parallelism.
> For a 2 layer topology Spout ---shuffle grouping--> bolt, when message size
> is small (around 100 bytes), we can find in the below picture that neither
> the CPU nor the network is saturated. When message size is 100 bytes, only
> 40% of CPU is used, only 18% of network is used, although we have a high
> parallelism (overall we have 144 executors)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
