Great, thanks for the info! One thing I've noticed is that the storm configuration isn't the full config as provided on topology deployment, it's just the 'base' config as specified in storm.yaml. It also doesn't have any sort of TopologyContext object provided. I'll get some more info, then open a bug report.
Simon From: Bobby Evans <[email protected]> Sent: 30 March 2018 14:56 To: [email protected] Subject: Re: Dynamic metrics in V2 system This should work for the most part. The new metrics API is just a thin wrapper around the codahale metrics API that has no restrictions on when you can add or remove metrics. It is separate from storm and it really is just a naming convention that is used to store which worker, component, etc, a metric is associated with. We have not exposed any of the remove APIs so if this is something you really need please file a JIRA for it and we can look into supporting it too. Thanks, Bobby On Thu, Mar 29, 2018 at 11:46 AM Simon Cooper <[email protected]<mailto:[email protected]>> wrote: Thanks! Does the new metrics system only create one reporter for the topology, and send the info between workers, or have one reporter per worker? Simon From: Jungtaek Lim <[email protected]<mailto:[email protected]>> Sent: 29 March 2018 16:12 To: [email protected]<mailto:[email protected]> Subject: Re: Dynamic metrics in V2 system Hi Simon, Actually I'm not familiar with codahale-based metrics system, but from the guide on codahale website, metrics can be registered even after starting reporters, which makes possible to have dynamic metrics. Please refer "Complete getting started" in https://metrics.dropwizard.io/3.1.0/getting-started/<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmetrics.dropwizard.io%2F3.1.0%2Fgetting-started%2F&data=02%7C01%7Csimon.cooper%40featurespace.co.uk%7C08937e98d90444458f4308d595876c84%7C19e863aab068484d9f9f990b545c5a0f%7C0%7C0%7C636579331230445822&sdata=58rMUWDUStdFsVjOtgikj%2BX%2BrD5nisGCmZBbqVSaQbg%3D&reserved=0> Hope this helps. Thanks, Jungtaek Lim (HeartSaVioR) 2018년 3월 28일 (수) 오후 9:38, Simon Cooper <[email protected]<mailto:[email protected]>>님이 작성: Hi, We're looking at upgrading our topologies to use the new V2 metrics system, however one of our absolutely core requirements is the ability to dynamically create/assign metrics after the topology is started up - we don't know what metrics we'll be running beforehand, and the metrics can change during the lifetime of the topology. We handled this in the V1 system using IMetric wrappers, but the codehale-based metrics system doesn't seem to have the ability to do this. If the V1 system is removed in a future version, this means we won't be able to use the storm metric system to report our metrics, which is a real shame, as we’ll lose all the built-in info that storm provides. We'll need to implement our own system from scratch & basically duplicate all the existing infrastructure. Is there a way we can create metrics dynamically in the V2 system? Thanks, Simon Cooper This message, and any files/attachments transmitted together with it, is intended for the use only of the person (or persons) to whom it is addressed. It may contain information which is confidential and/or protected by legal privilege. Accordingly, any dissemination, distribution, copying or use of this message, or any part of it or anything sent together with it, other than by intended recipients, may constitute a breach of civil or criminal law and is hereby prohibited. Unless otherwise stated, any views expressed in this message are those of the person sending it and not the sender's employer. No responsibility, legal or otherwise, of whatever nature, is accepted as to the accuracy of the contents of this message or for the completeness of the message as received. Anyone who is not the intended recipient of this message is advised to make no use of it and is requested to contact Featurespace Limited as soon as possible. Any recipient of this message who has knowledge or suspects that it may have been the subject of unauthorised interception or alteration is also requested to contact Featurespace Limited.
