Github user HeartSaVioR commented on the issue:
https://github.com/apache/storm/pull/2261
Now I have another numbers to persuade this patch.
I just take same approach to what @Ethanlm is done with his patch #2270
Performance testing on ConstSpoutNullBoltTopo with ACKing disabled.
1. Config: topology.message.timeout: 300; topology.max.spout.pending: 5000;
topology.acker.executors: 0
1. 1 VM from AWS c4.xlarge, `dedicated` to get more accurate result
1. Launched 1 workers, 1 spout task and 1 bolt task. ACKing disabled.
1. All experiments ran 300s.
1. For clarity, only show the outputs at 240s.
1. tested 3 times for each case, and pick one which showed median result
1. Numbers fluctuate slightly during the experiments.
Used 08038b6 (last commit) for this patch, and 77354fe for baseline
(master).
Grouping | transferred (messages) | transfer rate (message/s) |
spout_transferred | spout_acks | spout_throughput (acks/s)
-- | -- | -- | -- | -- | --
New LocalOrShuffle (patch) | 167984520 | 2799742 | 167984520 | 167984520 |
2799742
LocalOrShuffle (master) | 130891240 | 2181520 | 130891240 | 130891260 |
2181520
LocalOrShuffle with loadaware disabled (master) | 161410760 | 2690179 |
161410760 | 161410740 | 2690179
So the new LoadAwareShuffleGrouping is definitely faster than current
LoadAwareShuffleGrouping (about 28%), and *even faster* than current
ShuffleGrouping (about 4%).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
