On 1/31/14 3:58 AM, Tim Landscheidt wrote:
Antoine Musso <[email protected]> wrote:

I assume Ryan meant
https://wikitech.wikimedia.org/wiki/Special:NovaInstance
which has a column "Puppet status" ("ok"/"failed"/"stale").
That is nice!
Would it be possible to get notifications / emails whenever the status
of puppet change on an instance?  Or maybe a daily report would be enough.
I believe Puppet status was monitored by
http://icinga.wmflabs.org/ along with basic stuff like disk
usage & Co., but it has been unavailable for some time now.
<snip>

Sorry I'm late to this conversation. Indeed we use icinga on production to monitor puppet failures, and this used to work on labs. The implementation of this seems generally unpopular, so when I added the 'puppet status' filed to wikitech I routed around icinga and used a puppet reporting mechanism that integrates with OpenStack. There are a few incremental changes, but the basic framework is in https://gerrit.wikimedia.org/r/#/c/100221/. The report and callback system is quite simple! So, Petr, if you want a custom/limited solution you might look into just writing a custom reporter.

Regarding notifications or reporting... there are a few ways we could approach this. Probably the best thing is to revive icinga reporting, possibly be extending labsstatus.rb. For now, though, I'd advise you to just have a look at wikitech if you're curious.

-Andrew


_______________________________________________
Labs-l mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/labs-l

Reply via email to