[Betacluster-alerts] ** PROBLEM alert - deployment-mira/Puppet errors is CRITICAL **

2017-08-19 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-mira Address: 10.68.20.135 State: CRITICAL Date/Time: Sun 20 Aug 05:21:54 UTC 2017 Additional Info: CRITICAL: 20.00% of data above the critical threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is WARNING **

2017-08-19 Thread shinken
Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: WARNING Date/Time: Sun 20 Aug 05:16:48 UTC 2017 Additional Info: WARNING: 40.00% of data above the warning threshold [1.0] ___

[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is CRITICAL **

2017-08-19 Thread shinken
Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: CRITICAL Date/Time: Sun 20 Aug 05:11:47 UTC 2017 Additional Info: CRITICAL: 20.00% of data above the critical threshold [10.0]

[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is WARNING **

2017-08-19 Thread shinken
Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: WARNING Date/Time: Sun 20 Aug 05:01:48 UTC 2017 Additional Info: WARNING: 80.00% of data above the warning threshold [1.0] ___

[Betacluster-alerts] ** RECOVERY alert - deployment-apertium02/Puppet errors is OK **

2017-08-19 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-apertium02 Address: 10.68.22.254 State: OK Date/Time: Sun 20 Aug 05:00:54 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is CRITICAL **

2017-08-19 Thread shinken
Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: CRITICAL Date/Time: Sun 20 Aug 04:56:47 UTC 2017 Additional Info: CRITICAL: 20.00% of data above the critical threshold [10.0]

[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is WARNING **

2017-08-19 Thread shinken
Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: WARNING Date/Time: Sun 20 Aug 04:46:48 UTC 2017 Additional Info: WARNING: 80.00% of data above the warning threshold [1.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-apertium02/Puppet errors is CRITICAL **

2017-08-19 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-apertium02 Address: 10.68.22.254 State: CRITICAL Date/Time: Sun 20 Aug 04:25:55 UTC 2017 Additional Info: CRITICAL: 60.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** RECOVERY alert - deployment-mira/Puppet errors is OK **

2017-08-19 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-mira Address: 10.68.20.135 State: OK Date/Time: Sun 20 Aug 03:35:51 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-imagescaler02/Puppet errors is OK **

2017-08-19 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-imagescaler02 Address: 10.68.18.233 State: OK Date/Time: Sun 20 Aug 03:31:31 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing

[Betacluster-alerts] ** RECOVERY alert - deployment-urldownloader/Puppet errors is OK **

2017-08-19 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-urldownloader Address: 10.68.16.135 State: OK Date/Time: Sun 20 Aug 03:00:46 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing

[Betacluster-alerts] ** PROBLEM alert - deployment-imagescaler02/Puppet errors is CRITICAL **

2017-08-19 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-imagescaler02 Address: 10.68.18.233 State: CRITICAL Date/Time: Sun 20 Aug 02:56:33 UTC 2017 Additional Info: CRITICAL: 66.67% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-mira/Puppet errors is CRITICAL **

2017-08-19 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-mira Address: 10.68.20.135 State: CRITICAL Date/Time: Sun 20 Aug 02:55:54 UTC 2017 Additional Info: CRITICAL: 50.00% of data above the critical threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - deployment-mathoid/Puppet errors is OK **

2017-08-19 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-mathoid Address: 10.68.23.236 State: OK Date/Time: Sun 20 Aug 02:20:07 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-ores-redis-01/Puppet errors is OK **

2017-08-19 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-ores-redis-01 Address: 10.68.22.248 State: OK Date/Time: Sun 20 Aug 02:02:16 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing

[Betacluster-alerts] ** PROBLEM alert - deployment-urldownloader/Puppet errors is CRITICAL **

2017-08-19 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-urldownloader Address: 10.68.16.135 State: CRITICAL Date/Time: Sun 20 Aug 01:55:47 UTC 2017 Additional Info: CRITICAL: 55.56% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-mathoid/Puppet errors is CRITICAL **

2017-08-19 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-mathoid Address: 10.68.23.236 State: CRITICAL Date/Time: Sun 20 Aug 01:45:06 UTC 2017 Additional Info: CRITICAL: 44.44% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-ores-redis-01/Puppet errors is CRITICAL **

2017-08-19 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-ores-redis-01 Address: 10.68.22.248 State: CRITICAL Date/Time: Sun 20 Aug 01:22:14 UTC 2017 Additional Info: CRITICAL: 20.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** RECOVERY alert - deployment-logstash2/Puppet errors is OK **

2017-08-19 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-logstash2 Address: 10.68.16.147 State: OK Date/Time: Sun 20 Aug 00:49:20 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-logstash2/Puppet errors is CRITICAL **

2017-08-19 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-logstash2 Address: 10.68.16.147 State: CRITICAL Date/Time: Sun 20 Aug 00:14:20 UTC 2017 Additional Info: CRITICAL: 50.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** RECOVERY alert - deployment-mira/Puppet errors is OK **

2017-08-19 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-mira Address: 10.68.20.135 State: OK Date/Time: Sun 20 Aug 00:04:53 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-mira/Puppet errors is CRITICAL **

2017-08-19 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-mira Address: 10.68.20.135 State: CRITICAL Date/Time: Sat 19 Aug 23:24:53 UTC 2017 Additional Info: CRITICAL: 50.00% of data above the critical threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - deployment-kafka01/Puppet errors is OK **

2017-08-19 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-kafka01 Address: 10.68.21.219 State: OK Date/Time: Sat 19 Aug 22:33:18 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-imagescaler02/Puppet errors is OK **

2017-08-19 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-imagescaler02 Address: 10.68.18.233 State: OK Date/Time: Sat 19 Aug 22:30:30 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing

[Betacluster-alerts] ** RECOVERY alert - deployment-logstash2/Puppet errors is OK **

2017-08-19 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-logstash2 Address: 10.68.16.147 State: OK Date/Time: Sat 19 Aug 22:18:20 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-imagescaler02/Puppet errors is CRITICAL **

2017-08-19 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-imagescaler02 Address: 10.68.18.233 State: CRITICAL Date/Time: Sat 19 Aug 21:55:31 UTC 2017 Additional Info: CRITICAL: 55.56% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-kafka01/Puppet errors is CRITICAL **

2017-08-19 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-kafka01 Address: 10.68.21.219 State: CRITICAL Date/Time: Sat 19 Aug 21:53:17 UTC 2017 Additional Info: CRITICAL: 33.33% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-logstash2/Puppet errors is CRITICAL **

2017-08-19 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-logstash2 Address: 10.68.16.147 State: CRITICAL Date/Time: Sat 19 Aug 21:43:19 UTC 2017 Additional Info: CRITICAL: 33.33% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** RECOVERY alert - deployment-sentry01/Puppet errors is OK **

2017-08-19 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-sentry01 Address: 10.68.19.148 State: OK Date/Time: Sat 19 Aug 16:39:37 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-pdf01/Puppet errors is OK **

2017-08-19 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-pdf01 Address: 10.68.16.73 State: OK Date/Time: Sat 19 Aug 15:42:25 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-ores-redis-01/Puppet errors is CRITICAL **

2017-08-19 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-ores-redis-01 Address: 10.68.22.248 State: CRITICAL Date/Time: Sat 19 Aug 15:26:16 UTC 2017 Additional Info: CRITICAL: 66.67% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** RECOVERY alert - deployment-jobrunner02/Puppet staleness is OK **

2017-08-19 Thread shinken
Notification Type: RECOVERY Service: Puppet staleness Host: deployment-jobrunner02 Address: 10.68.19.42 State: OK Date/Time: Sat 19 Aug 15:04:08 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [3600.0] ___ Betacluster-alerts mailing

[Betacluster-alerts] ** RECOVERY alert - deployment-jobrunner02/Free space - all mounts is OK **

2017-08-19 Thread shinken
Notification Type: RECOVERY Service: Free space - all mounts Host: deployment-jobrunner02 Address: 10.68.19.42 State: OK Date/Time: Sat 19 Aug 14:37:46 UTC 2017 Additional Info: OK: All targets OK ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-mira/Puppet errors is OK **

2017-08-19 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-mira Address: 10.68.20.135 State: OK Date/Time: Sat 19 Aug 14:03:52 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-mira/Puppet errors is CRITICAL **

2017-08-19 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-mira Address: 10.68.20.135 State: CRITICAL Date/Time: Sat 19 Aug 13:23:51 UTC 2017 Additional Info: CRITICAL: 40.00% of data above the critical threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** PROBLEM alert - deployment-mathoid/Puppet errors is CRITICAL **

2017-08-19 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-mathoid Address: 10.68.23.236 State: CRITICAL Date/Time: Sat 19 Aug 10:44:05 UTC 2017 Additional Info: CRITICAL: 44.44% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** RECOVERY alert - deployment-aqs01/Puppet errors is OK **

2017-08-19 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-aqs01 Address: 10.68.18.237 State: OK Date/Time: Sat 19 Aug 10:10:57 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - Graphite Labs/Mediawiki Error Rate is OK **

2017-08-19 Thread shinken
Notification Type: RECOVERY Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: OK Date/Time: Sat 19 Aug 08:50:49 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [1.0] ___ Betacluster-alerts

[Betacluster-alerts] ** PROBLEM alert - deployment-logstash2/Puppet errors is CRITICAL **

2017-08-19 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-logstash2 Address: 10.68.16.147 State: CRITICAL Date/Time: Sat 19 Aug 08:12:19 UTC 2017 Additional Info: CRITICAL: 22.22% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** RECOVERY alert - deployment-mediawiki05/App Server Main HTTP Response is OK **

2017-08-19 Thread shinken
Notification Type: RECOVERY Service: App Server Main HTTP Response Host: deployment-mediawiki05 Address: 10.68.22.21 State: OK Date/Time: Sat 19 Aug 07:32:50 UTC 2017 Additional Info: HTTP OK: HTTP/1.1 200 OK - 50495 bytes in 0.847 second response time

[Betacluster-alerts] ** RECOVERY alert - deployment-ores-redis-01/Puppet errors is OK **

2017-08-19 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-ores-redis-01 Address: 10.68.22.248 State: OK Date/Time: Sat 19 Aug 07:30:14 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing

[Betacluster-alerts] ** RECOVERY alert - deployment-urldownloader/Puppet errors is OK **

2017-08-19 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-urldownloader Address: 10.68.16.135 State: OK Date/Time: Sat 19 Aug 07:29:46 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing

[Betacluster-alerts] ** RECOVERY alert - Generic Beta Cluster/English Wikipedia Mobile Main page is OK **

2017-08-19 Thread shinken
Notification Type: RECOVERY Service: English Wikipedia Mobile Main page Host: Generic Beta Cluster Address: en.wikipedia.beta.wmflabs.org State: OK Date/Time: Sat 19 Aug 07:26:33 UTC 2017 Additional Info: HTTP OK: HTTP/1.1 200 OK - 39217 bytes in 1.061 second response time

[Betacluster-alerts] ** PROBLEM alert - deployment-mediawiki05/App Server Main HTTP Response is CRITICAL **

2017-08-19 Thread shinken
Notification Type: PROBLEM Service: App Server Main HTTP Response Host: deployment-mediawiki05 Address: 10.68.22.21 State: CRITICAL Date/Time: Sat 19 Aug 07:23:00 UTC 2017 Additional Info: CRITICAL - Socket timeout after 10 seconds ___

[Betacluster-alerts] ** PROBLEM alert - Generic Beta Cluster/English Wikipedia Mobile Main page is CRITICAL **

2017-08-19 Thread shinken
Notification Type: PROBLEM Service: English Wikipedia Mobile Main page Host: Generic Beta Cluster Address: en.wikipedia.beta.wmflabs.org State: CRITICAL Date/Time: Sat 19 Aug 07:21:41 UTC 2017 Additional Info: CRITICAL - Socket timeout after 10 seconds

[Betacluster-alerts] ** RECOVERY alert - deployment-fluorine02/Free space - all mounts is OK **

2017-08-19 Thread shinken
Notification Type: RECOVERY Service: Free space - all mounts Host: deployment-fluorine02 Address: 10.68.23.106 State: OK Date/Time: Sat 19 Aug 07:19:59 UTC 2017 Additional Info: OK: All targets OK ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-ores-redis-01/Puppet errors is CRITICAL **

2017-08-19 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-ores-redis-01 Address: 10.68.22.248 State: CRITICAL Date/Time: Sat 19 Aug 06:55:15 UTC 2017 Additional Info: CRITICAL: 55.56% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-urldownloader/Puppet errors is CRITICAL **

2017-08-19 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-urldownloader Address: 10.68.16.135 State: CRITICAL Date/Time: Sat 19 Aug 06:54:47 UTC 2017 Additional Info: CRITICAL: 50.00% of data above the critical threshold [0.0] ___