[Betacluster-alerts] Host UP alert for beta-cluster!

2018-06-09 Thread shinken
Notification Type: RECOVERY Host: beta-cluster State: UP Address: en.wikipedia.beta.wmflabs.org Info: PING OK - Packet loss = 0%, RTA = 0.73 ms Date/Time: Sun 10 Jun 05:58:22 UTC 2018 ___ Betacluster-alerts mailing list

[Betacluster-alerts] Host DOWN alert for beta-cluster!

2018-06-09 Thread shinken
Notification Type: PROBLEM Host: beta-cluster State: DOWN Address: en.wikipedia.beta.wmflabs.org Info: check_ping: Invalid hostname/address - en.wikipedia.beta.wmflabs.org Date/Time: Sun 10 Jun 05:28:17 UTC 2018 ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-mediawiki06/Free space - all mounts is OK **

2018-06-09 Thread shinken
Notification Type: RECOVERY Service: Free space - all mounts Host: deployment-mediawiki06 Address: 10.68.19.241 State: OK Date/Time: Sun 10 Jun 01:29:58 UTC 2018 Notes URLs: Additional Info: OK: All targets OK ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-mediawiki06/Free space - all mounts is WARNING **

2018-06-09 Thread shinken
Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-mediawiki06 Address: 10.68.19.241 State: WARNING Date/Time: Sun 10 Jun 01:24:58 UTC 2018 Notes URLs: Additional Info: WARNING: deployment-prep.deployment-mediawiki06.diskspace.root.byte_percentfree (<11.11%)

[Betacluster-alerts] ** RECOVERY alert - deployment-mira/Puppet errors is OK **

2018-06-09 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-mira Address: 10.68.20.135 State: OK Date/Time: Sun 10 Jun 01:04:06 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - deployment-restbase01/Puppet errors is OK **

2018-06-09 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-restbase01 Address: 10.68.16.128 State: OK Date/Time: Sun 10 Jun 01:03:41 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - deployment-aqs03/Puppet errors is OK **

2018-06-09 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-aqs03 Address: 10.68.17.125 State: OK Date/Time: Sun 10 Jun 01:02:06 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - deployment-changeprop/Puppet errors is OK **

2018-06-09 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-changeprop Address: 10.68.16.88 State: OK Date/Time: Sun 10 Jun 01:02:10 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - deployment-aqs02/Puppet errors is OK **

2018-06-09 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-aqs02 Address: 10.68.17.90 State: OK Date/Time: Sun 10 Jun 01:02:18 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - deployment-aqs01/Puppet errors is OK **

2018-06-09 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-aqs01 Address: 10.68.18.237 State: OK Date/Time: Sun 10 Jun 01:01:59 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - deployment-parsoid09/Puppet errors is OK **

2018-06-09 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-parsoid09 Address: 10.68.20.142 State: OK Date/Time: Sun 10 Jun 01:01:09 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - deployment-sca01/Puppet errors is OK **

2018-06-09 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-sca01 Address: 10.68.20.183 State: OK Date/Time: Sun 10 Jun 01:01:00 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - deployment-mcs01/Puppet errors is OK **

2018-06-09 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-mcs01 Address: 10.68.17.18 State: OK Date/Time: Sun 10 Jun 01:00:11 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - deployment-imagescaler01/Puppet errors is OK **

2018-06-09 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-imagescaler01 Address: 10.68.19.158 State: OK Date/Time: Sun 10 Jun 01:00:13 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [0.0] ___

[Betacluster-alerts] ** RECOVERY alert - deployment-sca02/Puppet errors is OK **

2018-06-09 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-sca02 Address: 10.68.20.153 State: OK Date/Time: Sun 10 Jun 00:59:42 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - deployment-pdfrender02/Puppet errors is OK **

2018-06-09 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-pdfrender02 Address: 10.68.21.240 State: OK Date/Time: Sun 10 Jun 00:59:30 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [0.0] ___

[Betacluster-alerts] ** RECOVERY alert - deployment-tin/Puppet errors is OK **

2018-06-09 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-tin Address: 10.68.21.205 State: OK Date/Time: Sun 10 Jun 00:57:21 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - deployment-mediawiki06/Puppet errors is OK **

2018-06-09 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-mediawiki06 Address: 10.68.19.241 State: OK Date/Time: Sun 10 Jun 00:57:10 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [0.0] ___

[Betacluster-alerts] ** RECOVERY alert - deployment-mathoid/Puppet errors is OK **

2018-06-09 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-mathoid Address: 10.68.23.236 State: OK Date/Time: Sun 10 Jun 00:56:14 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** PROBLEM alert - deployment-aqs03/Puppet errors is CRITICAL **

2018-06-09 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-aqs03 Address: 10.68.17.125 State: CRITICAL Date/Time: Sat 09 Jun 23:37:05 UTC 2018 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-parsoid09/Puppet errors is CRITICAL **

2018-06-09 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-parsoid09 Address: 10.68.20.142 State: CRITICAL Date/Time: Sat 09 Jun 23:36:08 UTC 2018 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-logstash2/Puppet errors is CRITICAL **

2018-06-09 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-logstash2 Address: 10.68.16.147 State: CRITICAL Date/Time: Sat 09 Jun 23:33:56 UTC 2018 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-mcs01/Puppet errors is CRITICAL **

2018-06-09 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-mcs01 Address: 10.68.17.18 State: CRITICAL Date/Time: Sat 09 Jun 23:30:11 UTC 2018 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-pdfrender02/Puppet errors is CRITICAL **

2018-06-09 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-pdfrender02 Address: 10.68.21.240 State: CRITICAL Date/Time: Sat 09 Jun 23:29:31 UTC 2018 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0]

[Betacluster-alerts] ** PROBLEM alert - deployment-restbase01/Puppet errors is CRITICAL **

2018-06-09 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-restbase01 Address: 10.68.16.128 State: CRITICAL Date/Time: Sat 09 Jun 23:28:41 UTC 2018 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0]

[Betacluster-alerts] ** PROBLEM alert - deployment-zotero01/Puppet errors is CRITICAL **

2018-06-09 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-zotero01 Address: 10.68.17.102 State: CRITICAL Date/Time: Sat 09 Jun 23:27:36 UTC 2018 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-restbase02/Puppet errors is CRITICAL **

2018-06-09 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-restbase02 Address: 10.68.17.189 State: CRITICAL Date/Time: Sat 09 Jun 23:25:19 UTC 2018 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0]

[Betacluster-alerts] ** PROBLEM alert - deployment-mediawiki06/Puppet errors is CRITICAL **

2018-06-09 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-mediawiki06 Address: 10.68.19.241 State: CRITICAL Date/Time: Sat 09 Jun 23:22:09 UTC 2018 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0]

[Betacluster-alerts] ** PROBLEM alert - deployment-mathoid/Puppet errors is CRITICAL **

2018-06-09 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-mathoid Address: 10.68.23.236 State: CRITICAL Date/Time: Sat 09 Jun 23:21:12 UTC 2018 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-cassandra3-02/Puppet errors is CRITICAL **

2018-06-09 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-cassandra3-02 Address: 10.68.21.237 State: CRITICAL Date/Time: Sat 09 Jun 23:18:16 UTC 2018 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0]

[Betacluster-alerts] ** PROBLEM alert - deployment-tin/Puppet errors is CRITICAL **

2018-06-09 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-tin Address: 10.68.21.205 State: CRITICAL Date/Time: Sat 09 Jun 23:17:21 UTC 2018 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-aqs02/Puppet errors is CRITICAL **

2018-06-09 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-aqs02 Address: 10.68.17.90 State: CRITICAL Date/Time: Sat 09 Jun 23:17:19 UTC 2018 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-imagescaler01/Puppet errors is CRITICAL **

2018-06-09 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-imagescaler01 Address: 10.68.19.158 State: CRITICAL Date/Time: Sat 09 Jun 23:15:13 UTC 2018 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0]

[Betacluster-alerts] ** PROBLEM alert - deployment-mira/Puppet errors is CRITICAL **

2018-06-09 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-mira Address: 10.68.20.135 State: CRITICAL Date/Time: Sat 09 Jun 23:14:07 UTC 2018 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-aqs01/Puppet errors is CRITICAL **

2018-06-09 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-aqs01 Address: 10.68.18.237 State: CRITICAL Date/Time: Sat 09 Jun 23:11:58 UTC 2018 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-sca01/Puppet errors is CRITICAL **

2018-06-09 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-sca01 Address: 10.68.20.183 State: CRITICAL Date/Time: Sat 09 Jun 23:11:00 UTC 2018 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-cassandra3-01/Puppet errors is CRITICAL **

2018-06-09 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-cassandra3-01 Address: 10.68.17.103 State: CRITICAL Date/Time: Sat 09 Jun 23:08:35 UTC 2018 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0]

[Betacluster-alerts] ** PROBLEM alert - deployment-changeprop/Puppet errors is CRITICAL **

2018-06-09 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-changeprop Address: 10.68.16.88 State: CRITICAL Date/Time: Sat 09 Jun 23:07:10 UTC 2018 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-sca02/Puppet errors is CRITICAL **

2018-06-09 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-sca02 Address: 10.68.20.153 State: CRITICAL Date/Time: Sat 09 Jun 23:04:45 UTC 2018 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-cache-text04/Puppet errors is CRITICAL **

2018-06-09 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-cache-text04 Address: 10.68.18.103 State: CRITICAL Date/Time: Sat 09 Jun 21:43:24 UTC 2018 Notes URLs: Additional Info: CRITICAL: 66.67% of data above the critical threshold [0.0]

[Betacluster-alerts] ** RECOVERY alert - deployment-cache-text04/Puppet errors is OK **

2018-06-09 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-cache-text04 Address: 10.68.18.103 State: OK Date/Time: Sat 09 Jun 21:32:23 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-cache-text04/Puppet errors is CRITICAL **

2018-06-09 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-cache-text04 Address: 10.68.18.103 State: CRITICAL Date/Time: Sat 09 Jun 21:12:24 UTC 2018 Notes URLs: Additional Info: CRITICAL: 55.56% of data above the critical threshold [0.0]

[Betacluster-alerts] ** PROBLEM alert - deployment-deploy-01/SSH is CRITICAL **

2018-06-09 Thread shinken
Notification Type: PROBLEM Service: SSH Host: deployment-deploy-01 Address: 10.68.22.177 State: CRITICAL Date/Time: Sat 09 Jun 20:09:47 UTC 2018 Notes URLs: Additional Info: Connection refused ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - Graphite Labs/Mediawiki Error Rate is OK **

2018-06-09 Thread shinken
Notification Type: RECOVERY Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: OK Date/Time: Sat 09 Jun 18:59:46 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [1.0] ___

[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is WARNING **

2018-06-09 Thread shinken
Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: WARNING Date/Time: Sat 09 Jun 18:54:46 UTC 2018 Notes URLs: Additional Info: WARNING: 20.00% of data above the warning threshold [1.0]

[Betacluster-alerts] ** PROBLEM alert - deployment-puppetmaster03/Long lived cherry-picks on puppetmaster is CRITICAL **

2018-06-09 Thread shinken
Notification Type: PROBLEM Service: Long lived cherry-picks on puppetmaster Host: deployment-puppetmaster03 Address: 10.68.23.29 State: CRITICAL Date/Time: Sat 09 Jun 16:09:36 UTC 2018 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0]

[Betacluster-alerts] ** PROBLEM alert - deployment-maps03/Puppet errors is CRITICAL **

2018-06-09 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-maps03 Address: 10.68.18.91 State: CRITICAL Date/Time: Sat 09 Jun 13:18:45 UTC 2018 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-cpjobqueue/Puppet errors is CRITICAL **

2018-06-09 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-cpjobqueue Address: 10.68.22.161 State: CRITICAL Date/Time: Sat 09 Jun 09:07:23 UTC 2018 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0]

[Betacluster-alerts] ** PROBLEM alert - deployment-tin/Free space - all mounts is WARNING **

2018-06-09 Thread shinken
Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-tin Address: 10.68.21.205 State: WARNING Date/Time: Sat 09 Jun 08:51:10 UTC 2018 Notes URLs: Additional Info: WARNING: deployment-prep.deployment-tin.diskspace._mnt.byte_percentfree (No valid datapoints