Repository: storm Updated Branches: refs/heads/master c47c23cf8 -> ad63cc20f
STORM-2963 Adding notes on GC & single worker mode to Performance.md Project: http://git-wip-us.apache.org/repos/asf/storm/repo Commit: http://git-wip-us.apache.org/repos/asf/storm/commit/50bcb3c2 Tree: http://git-wip-us.apache.org/repos/asf/storm/tree/50bcb3c2 Diff: http://git-wip-us.apache.org/repos/asf/storm/diff/50bcb3c2 Branch: refs/heads/master Commit: 50bcb3c20a2c7ef77fbeac47ed8354b586b43d99 Parents: c367f9a Author: roshannaik <naikr...@gmail.com> Authored: Sun Sep 30 03:21:02 2018 -0700 Committer: roshannaik <naikr...@gmail.com> Committed: Sun Sep 30 03:22:09 2018 -0700 ---------------------------------------------------------------------- docs/Performance.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/storm/blob/50bcb3c2/docs/Performance.md ---------------------------------------------------------------------- diff --git a/docs/Performance.md b/docs/Performance.md index 20acb32..9738fef 100644 --- a/docs/Performance.md +++ b/docs/Performance.md @@ -163,3 +163,17 @@ Executors that are not expected to be busy can be allocated a smaller fraction o core for executors that are not likely to saturate the CPU. The *system bolt* generally processes very few messages per second, and so requires very little cpu (typically less than 10% of a physical core). + + +## 9. Garbage Collection +Choice of GC is an important concern for topologies that are latency or throughput sensitive. It is recommended to try the both the CMS and G1 collectors. Performance characteristics +of the collectors can change between single and multiworker modes and is dependent on hardware characteristics such as number of CPUs and memory localities. Number of GC threads can +also affect performance. Sometimes fewer GC threads can yield better performance. It is advisable to select a collector and tune it by mimicking anticipated peak data rates on hardware +similar to what is used in production. + + +## 10. Scaling out with Single Worker mode +Communication between executors within a worker process is very fast as there is neither a need to serialize and deserialize messages nor does it involve communicating over the network +stack. In multiworker mode, messages often cross worker process boundaries. For performance sensitive cases, if it is possible to configure a topology to run as many single-worker +instances (for ex. one worker per input partition) rather than one multiworker instance, it may yield significantly better throughput and latency on the same hardware. +The downside to this approach is that it adds the overhead of monitoring and managing many instances rather than one multiworker instance.
