I have not received any response and I cannot see how I could force curl_json plugin to organize the metric/values differently.
Is there a way in collectd to do a "global replace" where I change metric/value names? Given today's structure of data sent to Graphite: Metrics \---collectd \---serverA \---curl_json-serviceMetrics gauge-serverA-service1-requests gauge-serverA-service1-successes ... other 8 for service1 gauge-serverA-service2-requests ... other 9 for service2 ... and on for service3 - service10 I'd like to end up with: Metrics \---collectd \---serverA \---curl_json-serviceMetrics \---service1 gauge-serverA-service1-requests gauge-serverA-service1-successes ... other 8 for service1 \---service2 gauge-serverA-service2-requests ... other 9 for service2 ... and on for service3 - service10 In other words, I'd like to "inject" the service1 to service10 into the structure/hierarchy. I feel that collectd's filter mechanism could be used for this but I cannot see a clear way of doing it. Any suggestions? Thank you, dudlo ---------------------------- Original Message ---------------------------- Subject: curl_json configuration question (hierarchy) From: "Dudlo" <colle...@mozog.com> Date: Fri, December 4, 2015 12:34 To: collectd@verplant.org -------------------------------------------------------------------------- I am using the most recent curl_json plugin for collectd. It works but I'd like to configure it to generate more 'hierarchical' data. I have a server that returns some performance date in this (pseudo-code) JSON: { "serverA": { "service1": { "requests":"17", "successes":"15", "failures":"2", ... and 7 other values ... } "service2": { "requests":"28", ... and 9 other values ... } ... "service20" { ... } } } My collectd.conf looks like: LoadPlugin curl_json <Plugin curl_json> <URL "http://myserver/serviceMetrics"> Instance "serviceMetrics" <Key "*/*/requests"> Type "gauge" </Key> <Key "*/*/successes"> Type "gauge" </Key> ... and some 8 other specs </URL> </Plugin> When pushed to Graphite, this generates a "flat" list of 200 values (20 services, 10 values per service). The graphite/whisper data structure looks like: Metrics \---collectd \---serverA \---curl_json-serviceMetrics gauge-serverA-service1-requests gauge-serverA-service1-successes ... other 8 for service1 gauge-serverA-service2-requests ... other 9 for service2 ... and on for service3 - service10 This is nice for a first pass and would work but it is hard to manage to choose from 200+ items on the flat list when graphing. I would really like a deeper hierarchy sliced by the service: Metrics \---collectd \---serverA \---curl_json-serviceMetrics +---service1 | gauge-serverA-service1-requests | gauge-serverA-service1-successes | ... other 8 for service1 +---service2 | gauge-serverA-service2-requests | ... other 9 for service2 ... and on for service3 - service10 I do not know how to achieve it. I see: *** Option 1 - doable but unpractical. Change the collectd.conf to: <Plugin curl_json> <URL "http://myserver/serviceMetrics"> Instance "serviceMetrics-service1" <Key "*/service1/requests"> Type "gauge" </Key> <Key "*/service1/successes"> Type "gauge" </Key> ... and some 8 other specs </URL> <URL "http://myserver/serviceMetrics"> Instance "serviceMetrics-service2" <Key "*/service2/requests"> Type "gauge" </Key> <Key "*/service2/successes"> Type "gauge" </Key> ... and some 8 other specs </URL> ... and 8 more sections for service3 - service10 </Plugin> There are two issues: 1. The services are dynamic. There may be 5 or 25 of them and there are no name patterns in there (unlike the pseud-code in my example here). So I would have to update collectd.conf with every service addition/deletion/name change. 2. It is expensive - I'd have to redundantly call the serviceMetrics web service 20 times (for each service), every time reading only one service section. *** Option 2 - impossible with curl_json plugin. Ideally, I'd like to "force" the hierarchy on the value using the Instance clause inside the <Key>, something like: <Plugin curl_json> <URL "http://myserver/serviceMetrics"> Instance "serviceMetrics" <Key "*/*/requests"> Type "gauge" Instance "*/requests" </Key> <Key "*/*/successes"> Type "gauge" Instance "*/successes" </Key> ... and some 8 other specs </URL> </Plugin> The 'Instance "*/requests" ' clause does not work. It simply sets the name to the literal string (replacing special characters with underscores). In a way, I would like to use the concept of "capturing group" as in regex replacement. Sadly, curl_json is not ready for it. I even tried changing the web service definition to use multi-value type, as in load type (in types.db, load type is defined as "shortterm:GAUGE:0:5000, midterm:GAUGE:0:5000, longterm:GAUGE:0:5000"). Again, curl_json is not ready for such data structure - it expects a single numeric value. Is there a way I could coerce curl_json into generating values in a hierarchy? Thank you, dudlo _______________________________________________ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd