Ori.livneh has submitted this change and it was merged.
Change subject: Add alerting for MediaWiki exceptions and fatals
......................................................................
Add alerting for MediaWiki exceptions and fatals
Watch the rate per minute of MediaWiki exceptions and fatals. Warn if it
exceeds 25; panic if it exceeds 50. I hope we can gradually make these numbers
lower, but for now we need to make sure we're not too noisy or we'll
inadvertantly train people to ignore the alert.
Bug: T140942
Change-Id: I638d270e52a559a5b6bc0f68788172869ca2d888
---
M modules/role/manifests/graphite/alerts.pp
1 file changed, 8 insertions(+), 0 deletions(-)
Approvals:
Ori.livneh: Looks good to me, approved
Jforrester: Looks good to me, but someone else must approve
jenkins-bot: Verified
diff --git a/modules/role/manifests/graphite/alerts.pp
b/modules/role/manifests/graphite/alerts.pp
index c3031ca..c1c9f93 100644
--- a/modules/role/manifests/graphite/alerts.pp
+++ b/modules/role/manifests/graphite/alerts.pp
@@ -46,5 +46,13 @@
percentage => 40,
}
+ # Monitor MediaWiki fatals and exceptions.
+ monitoring::graphite_threshold { 'mediawiki_error_rate':
+ description => 'MediaWiki exceptions and fatals per minute',
+ metric =>
'transformNull(sumSeries(logstash.rate.mediawiki.fatal.ERROR.sum,
logstash.rate.mediawiki.exception.ERROR.sum), 0)',
+ warning => 25,
+ critical => 50,
+ from => '5min',
+ }
}
--
To view, visit https://gerrit.wikimedia.org/r/300327
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I638d270e52a559a5b6bc0f68788172869ca2d888
Gerrit-PatchSet: 5
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Ori.livneh <[email protected]>
Gerrit-Reviewer: Chad <[email protected]>
Gerrit-Reviewer: Greg Grossmeier <[email protected]>
Gerrit-Reviewer: Jforrester <[email protected]>
Gerrit-Reviewer: Ori.livneh <[email protected]>
Gerrit-Reviewer: jenkins-bot <>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
