[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is WARNING **

Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: WARNING Date/Time: Thu 31 Aug 05:51:49 UTC 2017 Additional Info: WARNING: 80.00% of data above the warning threshold [1.0] ___

[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is CRITICAL **

Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: CRITICAL Date/Time: Thu 31 Aug 05:41:51 UTC 2017 Additional Info: CRITICAL: 40.00% of data above the critical threshold [10.0]

[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is WARNING **

Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: WARNING Date/Time: Thu 31 Aug 04:51:48 UTC 2017 Additional Info: WARNING: 80.00% of data above the warning threshold [1.0] ___

[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is CRITICAL **

Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: CRITICAL Date/Time: Thu 31 Aug 04:46:49 UTC 2017 Additional Info: CRITICAL: 20.00% of data above the critical threshold [10.0]

[Betacluster-alerts] ** PROBLEM alert - deployment-fluorine02/Free space - all mounts is WARNING **

Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-fluorine02 Address: 10.68.23.106 State: WARNING Date/Time: Thu 31 Aug 03:58:02 UTC 2017 Additional Info: WARNING: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree (<33.33%)

[Betacluster-alerts] beta-scap-eqiad - Build # 171036 - Fixed!

beta-scap-eqiad - Build # 171036 - Fixed: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/171036/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] beta-scap-eqiad - Build # 171035 - Failure!

beta-scap-eqiad - Build # 171035 - Failure: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/171035/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** RECOVERY alert - deployment-kafka01/Puppet staleness is OK **

Notification Type: RECOVERY Service: Puppet staleness Host: deployment-kafka01 Address: 10.68.21.219 State: OK Date/Time: Thu 31 Aug 01:29:33 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [3600.0] ___ Betacluster-alerts mailing

[Betacluster-alerts] ** PROBLEM alert - deployment-kafka01/Puppet staleness is WARNING **

Notification Type: PROBLEM Service: Puppet staleness Host: deployment-kafka01 Address: 10.68.21.219 State: WARNING Date/Time: Thu 31 Aug 00:24:34 UTC 2017 Additional Info: WARNING: 50.00% of data above the warning threshold [3600.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-tin/Free space - all mounts is WARNING **

Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-tin Address: 10.68.21.205 State: WARNING Date/Time: Wed 30 Aug 21:04:14 UTC 2017 Additional Info: WARNING: deployment-prep.deployment-tin.diskspace._mnt.byte_percentfree (No valid datapoints

[Betacluster-alerts] ** RECOVERY alert - deployment-kafka01/Puppet staleness is OK **

Notification Type: RECOVERY Service: Puppet staleness Host: deployment-kafka01 Address: 10.68.21.219 State: OK Date/Time: Wed 30 Aug 20:03:32 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [3600.0] ___ Betacluster-alerts mailing

[Betacluster-alerts] ** PROBLEM alert - deployment-kafka01/Puppet staleness is WARNING **

Notification Type: PROBLEM Service: Puppet staleness Host: deployment-kafka01 Address: 10.68.21.219 State: WARNING Date/Time: Wed 30 Aug 19:53:32 UTC 2017 Additional Info: WARNING: 11.11% of data above the warning threshold [3600.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-imagescaler01/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-imagescaler01 Address: 10.68.19.158 State: CRITICAL Date/Time: Wed 30 Aug 19:30:38 UTC 2017 Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-ms-be04/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-ms-be04 Address: 10.68.16.139 State: CRITICAL Date/Time: Wed 30 Aug 19:20:53 UTC 2017 Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-cache-text04/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-cache-text04 Address: 10.68.18.103 State: CRITICAL Date/Time: Wed 30 Aug 19:20:48 UTC 2017 Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** RECOVERY alert - deployment-tin/Free space - all mounts is OK **

Notification Type: RECOVERY Service: Free space - all mounts Host: deployment-tin Address: 10.68.21.205 State: OK Date/Time: Wed 30 Aug 19:18:14 UTC 2017 Additional Info: OK: deployment-prep.deployment-tin.diskspace._mnt.byte_percentfree (No valid datapoints found)

[Betacluster-alerts] ** PROBLEM alert - deployment-ms-be03/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-ms-be03 Address: 10.68.22.125 State: CRITICAL Date/Time: Wed 30 Aug 19:17:31 UTC 2017 Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-tin/Free space - all mounts is WARNING **

Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-tin Address: 10.68.21.205 State: WARNING Date/Time: Wed 30 Aug 19:13:15 UTC 2017 Additional Info: WARNING: deployment-prep.deployment-tin.diskspace._mnt.byte_percentfree (No valid datapoints

[Betacluster-alerts] ** PROBLEM alert - deployment-ms-fe02/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-ms-fe02 Address: 10.68.19.247 State: CRITICAL Date/Time: Wed 30 Aug 19:12:19 UTC 2017 Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-cache-upload04/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-cache-upload04 Address: 10.68.18.109 State: CRITICAL Date/Time: Wed 30 Aug 19:09:05 UTC 2017 Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** RECOVERY alert - deployment-tin/Free space - all mounts is OK **

Notification Type: RECOVERY Service: Free space - all mounts Host: deployment-tin Address: 10.68.21.205 State: OK Date/Time: Wed 30 Aug 18:37:15 UTC 2017 Additional Info: OK: deployment-prep.deployment-tin.diskspace._mnt.byte_percentfree (No valid datapoints found)

[Betacluster-alerts] ** RECOVERY alert - deployment-kafka01/Puppet staleness is OK **

Notification Type: RECOVERY Service: Puppet staleness Host: deployment-kafka01 Address: 10.68.21.219 State: OK Date/Time: Wed 30 Aug 18:32:33 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [3600.0] ___ Betacluster-alerts mailing

[Betacluster-alerts] ** PROBLEM alert - deployment-tin/Free space - all mounts is WARNING **

Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-tin Address: 10.68.21.205 State: WARNING Date/Time: Wed 30 Aug 18:27:14 UTC 2017 Additional Info: WARNING: deployment-prep.deployment-tin.diskspace._mnt.byte_percentfree (No valid datapoints

[Betacluster-alerts] ** RECOVERY alert - deployment-mediawiki05/Free space - all mounts is OK **

Notification Type: RECOVERY Service: Free space - all mounts Host: deployment-mediawiki05 Address: 10.68.22.21 State: OK Date/Time: Wed 30 Aug 18:25:09 UTC 2017 Additional Info: OK: All targets OK ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-mediawiki05/Free space - all mounts is WARNING **

Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-mediawiki05 Address: 10.68.22.21 State: WARNING Date/Time: Wed 30 Aug 18:20:09 UTC 2017 Additional Info: WARNING: deployment-prep.deployment-mediawiki05.diskspace.root.byte_percentfree (<11.11%)

[Betacluster-alerts] ** PROBLEM alert - deployment-kafka01/Puppet staleness is WARNING **

Notification Type: PROBLEM Service: Puppet staleness Host: deployment-kafka01 Address: 10.68.21.219 State: WARNING Date/Time: Wed 30 Aug 17:52:34 UTC 2017 Additional Info: WARNING: 33.33% of data above the warning threshold [3600.0] ___

[Betacluster-alerts] ** RECOVERY alert - deployment-kafka01/Puppet staleness is OK **

Notification Type: RECOVERY Service: Puppet staleness Host: deployment-kafka01 Address: 10.68.21.219 State: OK Date/Time: Wed 30 Aug 16:31:33 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [3600.0] ___ Betacluster-alerts mailing

[Betacluster-alerts] ** PROBLEM alert - deployment-puppetmaster02/Long lived cherry-picks on puppetmaster is CRITICAL **

Notification Type: PROBLEM Service: Long lived cherry-picks on puppetmaster Host: deployment-puppetmaster02 Address: 10.68.21.200 State: CRITICAL Date/Time: Wed 30 Aug 16:25:09 UTC 2017 Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0]

[Betacluster-alerts] ** RECOVERY alert - deployment-tin/Free space - all mounts is OK **

Notification Type: RECOVERY Service: Free space - all mounts Host: deployment-tin Address: 10.68.21.205 State: OK Date/Time: Wed 30 Aug 09:41:15 UTC 2017 Additional Info: OK: deployment-prep.deployment-tin.diskspace._mnt.byte_percentfree (No valid datapoints found)

[Betacluster-alerts] ** PROBLEM alert - deployment-tin/Free space - all mounts is WARNING **

Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-tin Address: 10.68.21.205 State: WARNING Date/Time: Wed 30 Aug 09:31:15 UTC 2017 Additional Info: WARNING: deployment-prep.deployment-tin.diskspace._mnt.byte_percentfree (No valid datapoints

[Betacluster-alerts] ** RECOVERY alert - Graphite Labs/Mediawiki Error Rate is OK **

Notification Type: RECOVERY Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: OK Date/Time: Wed 30 Aug 08:55:47 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [1.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - deployment-kafka01/Puppet staleness is OK **

Notification Type: RECOVERY Service: Puppet staleness Host: deployment-kafka01 Address: 10.68.21.219 State: OK Date/Time: Wed 30 Aug 07:30:32 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [3600.0] ___ Betacluster-alerts mailing

[Betacluster-alerts] ** RECOVERY alert - deployment-fluorine02/Free space - all mounts is OK **

Notification Type: RECOVERY Service: Free space - all mounts Host: deployment-fluorine02 Address: 10.68.23.106 State: OK Date/Time: Wed 30 Aug 07:12:00 UTC 2017 Additional Info: OK: All targets OK ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-kafka01/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-kafka01 Address: 10.68.21.219 State: CRITICAL Date/Time: Wed 30 Aug 06:57:17 UTC 2017 Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-fluorine02/Free space - all mounts is CRITICAL **

Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-fluorine02 Address: 10.68.23.106 State: CRITICAL Date/Time: Wed 30 Aug 06:37:01 UTC 2017 Additional Info: CRITICAL: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree (<20.00%)

[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is CRITICAL **

Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: CRITICAL Date/Time: Wed 30 Aug 06:19:49 UTC 2017 Additional Info: CRITICAL: 20.00% of data above the critical threshold [10.0]

[Betacluster-alerts] ** RECOVERY alert - deployment-mediawiki04/App Server Main HTTP Response is OK **

Notification Type: RECOVERY Service: App Server Main HTTP Response Host: deployment-mediawiki04 Address: 10.68.19.128 State: OK Date/Time: Wed 30 Aug 06:04:27 UTC 2017 Additional Info: HTTP OK: HTTP/1.1 200 OK - 50509 bytes in 1.197 second response time

[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is WARNING **

Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: WARNING Date/Time: Wed 30 Aug 05:59:49 UTC 2017 Additional Info: WARNING: 80.00% of data above the warning threshold [1.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-mediawiki04/App Server Main HTTP Response is CRITICAL **

Notification Type: PROBLEM Service: App Server Main HTTP Response Host: deployment-mediawiki04 Address: 10.68.19.128 State: CRITICAL Date/Time: Wed 30 Aug 05:59:37 UTC 2017 Additional Info: CRITICAL - Socket timeout after 10 seconds ___