GGraziadei commented on PR #8593: URL: https://github.com/apache/storm/pull/8593#issuecomment-4693418682
## Benchmark To validate this PR I executed a couple of benchmarks to be sure: - No throughput/latency degradation - A real advantage to use `JitterAwareStreamGrouping` in stressed env The benchmark is available in `storm-perf/JitterAwareGroupingTopology` I asked an LLM to prepare the attached report, given the benchmark outcome. Attached to this comment are also the raw data (storm-perf result + Grafana metrics) ``` (baseline) ------------------------------------------------------------------------------------------------------------------ elapsed (sec),workers,tasks,executors,transferred (messages),transfer rate (messages/s),spout_executors,spout_transferred (messages),spout_acks,spout_throughput (acks/s),spout_avg_complete_latency(ms),spout_max_complete_latency(ms) ------------------------------------------------------------------------------------------------------------------ 60,2,10,10,0,0,0,0,0,0,NaN,0.000 120,2,10,10,136220,2270,1,10180,4800,169,29085.146,29085.146 180,2,10,10,74860,1247,1,5060,1640,84,35036.022,35036.022 240,2,10,10,72340,1205,1,4580,1680,76,39128.936,39128.936 Grafana graphs: https://snapshots.raintank.io/dashboard/snapshot/9258h5bMeFXXgdCJ5pcRDZpewUvaGt0P (feedback control loop) ------------------------------------------------------------------------------------------------------------------ elapsed (sec),workers,tasks,executors,transferred (messages),transfer rate (messages/s),spout_executors,spout_transferred (messages),spout_acks,spout_throughput (acks/s),spout_avg_complete_latency(ms),spout_max_complete_latency(ms) ------------------------------------------------------------------------------------------------------------------ 60,2,10,10,0,0,0,0,0,0,NaN,0.000 120,2,10,10,212840,3547,1,15880,11960,264,16683.642,16683.642 180,2,10,10,156920,2615,1,10800,8800,180,16439.405,16683.642 240,2,10,10,42980,716,1,5140,1180,85,15821.700,16683.642 Grafana graphs: https://snapshots.raintank.io/dashboard/snapshot/t5D0t0X5FDbdUCUJut1bsdvTIIpdWepu ``` [Experimental Results- Impact of FeedbackTuple on Jitter-Aware Stream Grouping.pdf](https://github.com/user-attachments/files/28889773/Experimental.Results-.Impact.of.FeedbackTuple.on.Jitter-Aware.Stream.Grouping.pdf) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
