Github user Gvain commented on the pull request:
https://github.com/apache/incubator-storm/pull/103#issuecomment-44629767
@clockfly , @revans2
I added more nodes, from 4 nodes to 15 nodes, as well as spouts, bolts,
ackers and workers, and keep their parallelism count in the same ratio, eg, for
every other 4 nodes, add 48 spouts, 48 bolts, 48 ackers and 8 workers. Here is
the test result:
nodes | workers | Throughput | CPU usage | NET usage (only IN Bytes)
4 | 8 | 656,000 tps | 89% | 28MB/s
8 | 16 | 1,004,000 tps | 82% | 28MB/s
12 | 24 | 1,133,000 tps | 72% | 25MB/s
15 | 30 | 1,235,000 tps | 69% | 24MB/s
for last two rows, the CPU usage decreased, and the throughput increasing
speed slows down. I added a few more worker to re-run the test, the result is:
nodes | workers | Throughput | CPU usage | NET usage (only IN Bytes)
12 | 48 | 1,444,000 tps | 88% | 30MB/s
15 | 30 | 1,735,000 tps | 88% | 30MB/s
From the result, We can see adding some more workers DO help to scale up
performance as adding more CPUs.
Finally, maybe we should re-considerate the root cause of this issue
"STORM-297 Storm Performance cannot be scaled up by adding more CPU cores"
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
