Github user Ethanlm commented on the issue:
https://github.com/apache/storm/pull/2366
I ran some performance tests using ThroughputVsLatency and the raw results
are available at https://github.com/Ethanlm/STORM-2686-perf-study
The experiments include
| Experiment | Description
| ------------- |:-------------:|
| ORIGINAL | the current storm LoadAwareShuffleGrouping|
| SHUFFLE | the current storm with loadaware disabled |
| STORM-2686 | this patch with higher bound 0.8 and lower bound 0.2
| STORM-2686-H70L30| this patch with higher bound 0.7 and lower bound 0.3 |
Every experiment runs on 2 VMs with 2/4/8 workers and 5000/10000/20000 rate
separately. The number of spouts, splitters and counters are all equal to the
number of workers for simplicity. Repeat three times.
The table below shows the average of all the experiments (except for the
experiments with 8workers and 20000rate because they all failed)
| Experiment | rate(tuple/s) | mean(ms) | 99%ile(ms) | 99.9%ile(ms) | cores
| mem(MB) |
| ------------- |:-------------:| -----:|------------- |:-------------:|
-----:|-----:|
| ORIGINAL | 10862.78696 | 11.12642261 | 16.76319565 | 25.67112174 | 2.1671
| 416.8698696 |
|SHUFFLE | 10625.18 | 11.15462583 | 16.94839583 | 25.39703333 | 2.174539167
| 369.2407083 |
|STORM-2686 | 10626.52542 | 10.40140958 | 15.68085 | 29.86079583 | 2.15822
| 378.7717542|
|STORM-2686-H70L30| 10624.41 | 10.51196542 | 15.7350125 | 26.97664583 |
2.136774583 | 411.7942958|
It seems that from the result this patch did bring some benefits especially
about overall 6.6% less on mean latency. If you look at the single experiment
results, this patch seems to do a better job than the original storm.
Since I didn't change the critical functions, the performance of `choose a
Task` should remain the same.
---
