[
https://issues.apache.org/jira/browse/STORM-297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14021328#comment-14021328
]
ASF GitHub Bot commented on STORM-297:
--------------------------------------
Github user miguno commented on the pull request:
https://github.com/apache/incubator-storm/pull/103#issuecomment-45448273
FYI: I'll add Nathan's comment on our test methodology to DEVELOPER.md.
> On 07.06.2014, at 20:59, Nathan Marz <[email protected]> wrote:
>
> -1 Tests should never, ever rely on timing in order to pass. This is the
whole reason for doing time simulation in the first place, so that when
functionality depends on time it can be properly tested without having to worry
about random delays messing up the tests.
>
> complete-topology is inherently reliant on detecting topology completion
based on the spout saying all its tuples are "complete". If you're testing
topologies that don't do full tuple acking, then you should be testing using
the "tracked topologies" utilities in backtype.storm.testing.clj
>
> For example, here is how the acking system is tested using tracked
topologies:
https://github.com/apache/incubator-storm/blob/master/storm-core/test/clj/backtype/storm/integration_test.clj#L213
>
> The "tracked-wait" function is the key which will only return when both
that many tuples have been emitted by the spouts AND the topology is idle (no
tuples have been emitted nor will be emitted without further input) You
shouldn't use tracked-topologies for topologies that have tick tuples, but that
shouldn't be a problem in this case.
>
> —
> Reply to this email directly or view it on GitHub.
> Storm Performance cannot be scaled up by adding more CPU cores
> --------------------------------------------------------------
>
> Key: STORM-297
> URL: https://issues.apache.org/jira/browse/STORM-297
> Project: Apache Storm (Incubating)
> Issue Type: Bug
> Reporter: Sean Zhong
> Labels: Performance, netty
> Fix For: 0.9.2-incubating
>
> Attachments: Storm_performance_fix.pdf,
> storm_Netty_receiver_diagram.png, storm_conf.txt,
> storm_performance_fix.patch, worker_throughput_without_storm-297.png
>
>
> We cannot scale up the performance by adding more CPU cores and increasing
> parallelism.
> For a 2 layer topology Spout ---shuffle grouping--> bolt, when message size
> is small (around 100 bytes), we can find in the below picture that neither
> the CPU nor the network is saturated. When message size is 100 bytes, only
> 40% of CPU is used, only 18% of network is used, although we have a high
> parallelism (overall we have 144 executors)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
