Thanks for the quick reply Jungtaek,
That clears things up for me. One other question. For the metrics related to sendqueue and receive, am I right in thinking these are related to the Disruptor send queue and Disruptor receive queue for the executor running each task and not the send and receive threads for each worker process? Thanks, Thomas Cooper PhD Student Newcastle University, School of Computer Science ________________________________ From: Jungtaek Lim <[email protected]> Sent: 26 May 2016 13:32 To: [email protected] Subject: Re: Clarification on metrics sampling rate Hi Tom, At first, user mailing list is more proper place since dev mailing list is for Storm developers (committers / PMCs / contributors) talking about improving/maintaining Storm. Topology built-in metrics are sampled regardless of consumer (UI metric - actually task heartbeat, or metrics consumer). While metrics provided to UI has time windows, metrics provided to metrics consumer resets their values every period. Hope this help. Thanks, Jungtaek Lim (HeartSaVioR) 2016? 5? 26? (?) ?? 9:11, Thomas Cooper (PGR) <[email protected]<mailto:[email protected]>>?? ??: Hi, I'm a Computer Science PhD student working on modelling the performance of distributed stream processing systems like Storm. I am attempting to use Queueing Theory to model the performance of a running topology and then make predictions about performance under varying input loads. To do this accurately I need metrics for the latency, arrival and emission rates of each task (among other things), which Storm happily gives me. However, I know that Storm samples the summary metrics for the UI using the Config.TOPOLOGY_STATS_SAMPLE_RATE value (default to 0.05), but does this apply to the metrics on the "__metrics" stream which, as I understand it, are sent to any bolt implementing IMetricConsumer registered with Topology? Any hints would be greatly appreciated, as a last resort I can go digging in the source code but I would like to avoid that if possible. Also let me know if this would be better posted on the dev mailing list. This is my 1st time using the mailing list, I am likely to have more questions in the future and I want to avoid spamming the wrong people. Thanks in advance, Tom Cooper PhD Student Newcastle University, School of Computer Science
