[
https://issues.apache.org/jira/browse/STORM-297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14003245#comment-14003245
]
ASF GitHub Bot commented on STORM-297:
--------------------------------------
Github user clockfly commented on the pull request:
https://github.com/apache/incubator-storm/pull/103#issuecomment-43627624
Gvain,
>What if we use MORE workers per node than just only ONE worker per node
without changing the total number of executors ? By doing so, we will have MORE
received threads, transfer threads and netty i/o threads for the total 144
executors. Should this increase total CPU usage and network bandwidth usage?
We tried this approach before, but it won't give us the performance data we
want. There were inherit bottleneck there. Besides, receiver.count is not the
biggest bottleneck here, netty performance matter more.
The logic behind make "receiver.count" configurable is that, since we allow
user to configure spout/bolt executor count per worker, we should also allow
user to configure the receiver thread count to make it consistent with the
parallism settings.
Besides, increasing worker count will,
1. add more cross process or cross machine communication. For tasks inside
same worker process, the message will be **local** dispatched to target task
if possible.
2. More Netty context switch and contention. Check
http://yahooeng.tumblr.com/post/64758709722/making-storm-fly-with-netty
3. More outbound acker message count. Usually we will allocate one acker to
one worker.
So in common practice, each worker will have a moderate size of executors,
neither too small, nor too big.
> Storm Performance cannot be scaled up by adding more CPU cores
> --------------------------------------------------------------
>
> Key: STORM-297
> URL: https://issues.apache.org/jira/browse/STORM-297
> Project: Apache Storm (Incubating)
> Issue Type: Bug
> Reporter: Sean Zhong
> Labels: Performance, netty
> Fix For: 0.9.2-incubating
>
> Attachments: Storm_performance_fix.pdf,
> storm_Netty_receiver_diagram.png, storm_performance_fix.patch
>
>
> We cannot scale up the performance by adding more CPU cores and increasing
> parallelism.
> For a 2 layer topology Spout ---shuffle grouping--> bolt, when message size
> is small (around 100 bytes), we can find in the below picture that neither
> the CPU nor the network is saturated. When message size is 100 bytes, only
> 40% of CPU is used, only 18% of network is used, although we have a high
> parallelism (overall we have 144 executors)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
