Thought you all might be interested. I have now got capacity monitoring working with AWS Cloudwatch. These are 2 bolts within the same topology:
​ #! /bin/bash API_KEY='XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' # get all the topology id's TOPOLOGY_IDS=`curl -s http://XXXXXXXXXXXXXXXXXXXXXXXX:8080/api/v1/topology/summary | json_pp | grep '"id"' | sed 's/.* : "\(.*\)".*/\1/'` for id in ${TOPOLOGY_IDS} ; do TOPOLOGY=`curl -s http://XXXXXXXXXXXXXXXXXXXXXXX:8080/api/v1/topology/${id}?sys=false` # check the capacity for each task CAPACITIES=`echo ${TOPOLOGY} | json_pp | grep '"capacity"' | sed 's/.* : "\(.*\)".*/\1/'` CAPACITIES_ARRAY=() for capactiy in ${CAPACITIES} ; do CAPACITIES_ARRAY+=("${capactiy}") done # get all of the bolt ids BOLT_IDS=`echo ${TOPOLOGY} | json_pp | grep '"boltId"' | sed 's/.* : "\(.*\)".*/\1/'` BOLT_IDS_ARRAY=() for boltId in ${BOLT_IDS} ; do BOLT_IDS_ARRAY+=("${boltId}") done # send a trigger if needed # capacity and bolt id arrays will always be the same size for(( x=0; x<${#CAPACITIES_ARRAY[@]}; x++ )); do TIMESTAMP=`date +"%Y-%m-%dT%H:%M:%S.%N"` /usr/local/bin/aws cloudwatch put-metric-data --metric-name "${BOLT_IDS_ARRAY[$x]}-capacity" --namespace "${id}" --value "${CAPACITIES_ARRAY[$x]}" --timestamp "${TIMESTAMP}" done done On Mon, May 30, 2016 at 5:37 PM, Radhwane Chebaane <[email protected] > wrote: > > Using graphite API was so helpful for us, but since it doesn't support > *Tags* introduced since *InfluxDB 0.9*, we created a *new metric Consumer > <https://github.com/mathieuboniface/storm-metrics-influxdb>* that > supports the new InfluxDB API instead of Grahpite API. *Tags* are > important when filtering dashboard metrics based on components, bolt or > worker name. > > Regards, > Radhwane > > > On 30/05/2016 17:19, Stephen Powis wrote: > > +1 for graphite and grafana via Verisign's plugin. > > Using graphite a few years ago was a real game changer for us, and more > recently grafana to help build out dashboards instead of copy/pasting > graphite urls around. Here's two different dashboards we have relating to > our storm topologies. We're able to correlate information from all parts > of our app, hardware monitoring metrics (via zabbix) and of course storm. > Additionally we use seyren on top of graphite for our alerting as well. > > bolt specific dashboard <http://i.imgur.com/ftKtci5.png> > > Dashboard correlating lots of related information from various sources > <http://i.imgur.com/t7yJ8d5.jpg> > > > > On Mon, May 30, 2016 at 9:13 AM, Radhwane Chebaane < > <[email protected]>[email protected]> wrote: > >> Hi Matthew, >> >> We actually use the InfluxData <https://influxdata.com/> Stack (InfluxDb >> + Grafana). >> We send our data directly to a time-series database, *InfluxDB* >> <https://influxdata.com/time-series-platform/influxdb/>. Then, we >> visualize metrics with a customizable dashboard, *Grafana* >> <http://grafana.org/>. >> This way, you can have real-time metrics on your Storm topology. You may >> also add custom metrics for enhanced monitoring. >> >> To export Storm metrics to InfluxDB you can use this *MetricsConsumer *which >> is compatible with the latest version of InfluxDB and Storm 1.0.0: >> https://github.com/mathieuboniface/storm-metrics-influxdb >> >> Or you can use the old Verisign plug-in with Graphite protocol: >> https://github.com/verisign/storm-graphite >> >> Best regards, >> Radhwane CHEBAANE >> >> >> On 30/05/2016 14:47, Matthew Lowe wrote: >> >> Hello all. >> >> What kind of monitoring solutions do you use with storm? >> >> For example I have a bash script that reads the Json data from the REST UI >> and alerts if there are any bolts with high capacities. >> >> It's only small and hacky, but I am genuinely interested to how you all >> monitor your topologies. >> >> Best Regards >> Matthew Lowe >> >> >> > >
