Harsha, Yes I'm thinking about pushing so Nimbus is a best place to add.
It would be different than MetricsConsumer due to, - Only leader of Nimbus should publish cluster metrics to consumer. - Nimbus shouldn't be affected by crashing or heavy latency on consumer. - Consumer should have resilient when crashing or Nimbus should take care of. I'll sketch the design and come up with JIRA issue or new thread on dev@. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 4월 20일 (수) 오전 1:23, Harsha <[email protected]>님이 작성: > Jungtaek, > Ideally we should avoid using NimbusClient as much as > possible for a polling reporter like this. Can we have > plug gable interface that can be run inside the nimbus at > configured interval . Since it will be plugged into nimbus > we would have all cluster details including topologies as > well. > > Thanks, > Harsha > > > On Mon, Apr 18, 2016, at 10:18 PM, Jungtaek Lim wrote: > > Cody, > > > > Great idea! Let's try it out. > > I'll send the mail to user@ regarding use cases or concerns of metrics > on > > Apache Storm. We may have a luck to find brilliant ideas or use cases we > > didn't know. > > > > 2016년 4월 19일 (화) 오후 2:07, Cody Innowhere <[email protected]>님이 작성: > > > > > Jungtaek, > > > Thanks for your explanation, still +1 for your proposal. > > > Moreover, I'd like to hear storm users about this, especially use > cases for > > > metrics so that we can see what's best to do this. > > > > > > On Tue, Apr 19, 2016 at 11:19 AM, Jungtaek Lim <[email protected]> > wrote: > > > > > > > Cody, > > > > Thanks for giving an opinion. > > > > > > > > I guess there was a miscommunication between you and me. > > > > This shouldn't be related to topology. Plugin should be set up via > > > cluster > > > > configuration, and singular daemon like Nimbus (master) or UI should > > > > publish cluster metrics to the plugin. > > > > I agree Nimbus should have this feature, but only master should > publish > > > the > > > > cluster metrics so we should handle it. > > > > > > > > Btw, since we don't have built-in metrics storage, we can't provide > query > > > > API with time-series fashion. > > > > I guess publishing (pushing) seems the better way what we can address > > > > without huge modification. > > > > (Did you mean Nimbus has query API which queries the external > storage?) > > > > > > > > Please note that it's an idea for improvement against 1.x, not 2.0. > > > > In phase 2 we should evaluate and replace metrics feature with > JStorm or > > > > another after porting to Java. > > > > > > > > Thanks, > > > > Jungtaek Lim (HeartSaVioR) > > > > > > > > 2016년 4월 19일 (화) 오전 11:59, Cody Innowhere <[email protected]>님이 > 작성: > > > > > > > > > I'm +1 for publishing cluster stats, but don't quite understand > your > > > > > implementation. > > > > > Do you mean that a topology which wants cluster stats can get > cluster > > > > stats > > > > > by implementing the pluggable consumer/reporter? > > > > > Personally I would prefer to put this role into nimbus (can be > > > pluggable > > > > > too) since it's the responsibility of nimbus to take care of all > > > > > topologies, naturally all topology stats. Then we expose API's from > > > > nimbus > > > > > for external queries. > > > > > > > > > > On Tue, Apr 19, 2016 at 9:47 AM, Jungtaek Lim <[email protected]> > > > wrote: > > > > > > > > > > > Hi devs, > > > > > > > > > > > > While Storm publishes topology metrics by metrics consumers, > Storm > > > > > doesn't > > > > > > publish cluster metrics any way. > > > > > > > > > > > > Therefore, I'd like to introduce the feature of publishing > cluster > > > > stats > > > > > to > > > > > > pluggable consumers so that users can also push to external > storages > > > > and > > > > > > query on it or even configure dashboard. > > > > > > (For topology stats it's supported via MetricsConsumer.) > > > > > > > > > > > > I'm seeing some workarounds like having their own reporter > process > > > > > polling > > > > > > cluster information from Nimbus or REST API. > > > > > > (For example, Ambari has cluster metrics reporter for Storm > > > > > > < > > > > > > > > > > > > > > > > > > > https://github.com/apache/ambari/blob/trunk/ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java > > > > > > > > > > > > > but it relies on custom build of Storm.) > > > > > > Yes it works anyway but if Storm provides the feature naturally, > with > > > > > > pluggable way like MetricsConsumer it would be great for users > who > > > want > > > > > to > > > > > > configure dashboard regarding this. > > > > > > > > > > > > I also saw that STORM-1158 < > > > > > > http://issues.apache.org/jira/browse/STORM-1158> > > > > > > adds metrics reporter for internal actions, but it has some > > > limitation > > > > on > > > > > > it. > > > > > > > > > > > > - It focuses how many requests the daemon receives, not cluster > > > stats. > > > > > > - It requires reporter to use codahale metrics. Moreover it > shades > > > > > > codahale-metrics so other implementations of reporter may not > work > > > > > properly > > > > > > if it uses other codahale-metrics plugin like ganglia. > > > > > > > > > > > > So I'm planning to add metrics feature of cluster stat by not > relying > > > > on > > > > > > daemon metrics reporter, but introduce new pluggable > > > consumer/reporter. > > > > > > > > > > > > What do you think? Do you have opinion that we need to add > cluster > > > > stats > > > > > to > > > > > > daemon metrics reporter? > > > > > > > > > > > > Thanks, > > > > > > Jungtaek Lim (HeartSaVioR) > > > > > > > > > > > > > > > > > > >
