[Betacluster-alerts] beta-scap-eqiad - Build # 162458 - Failure!

beta-scap-eqiad - Build # 162458 - Failure: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/162458/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** PROBLEM alert - deployment-ores-redis-01/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-ores-redis-01 Address: 10.68.22.248 State: CRITICAL Date/Time: Tue 04 Jul 00:23:43 UTC 2017 Additional Info: CRITICAL: 40.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] beta-scap-eqiad - Build # 162446 - Failure!

beta-scap-eqiad - Build # 162446 - Failure: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/162446/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] beta-scap-eqiad - Build # 162447 - Fixed!

beta-scap-eqiad - Build # 162447 - Fixed: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/162447/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] beta-scap-eqiad - Build # 162459 - Fixed!

beta-scap-eqiad - Build # 162459 - Fixed: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/162459/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** PROBLEM alert - deployment-pdfrender02/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-pdfrender02 Address: 10.68.21.240 State: CRITICAL Date/Time: Tue 04 Jul 03:18:30 UTC 2017 Additional Info: CRITICAL: 60.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** RECOVERY alert - deployment-tin/Free space - all mounts is OK **

Notification Type: RECOVERY Service: Free space - all mounts Host: deployment-tin Address: 10.68.21.205 State: OK Date/Time: Mon 03 Jul 20:13:35 UTC 2017 Additional Info: OK: deployment-prep.deployment-tin.diskspace._mnt.byte_percentfree (No valid datapoints found)

[Betacluster-alerts] beta-scap-eqiad - Build # 162406 - Fixed!

beta-scap-eqiad - Build # 162406 - Fixed: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/162406/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** PROBLEM alert - deployment-sca04/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-sca04 Address: 10.68.18.80 State: CRITICAL Date/Time: Mon 03 Jul 20:20:42 UTC 2017 Additional Info: CRITICAL: 50.00% of data above the critical threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - deployment-mathoid/Puppet errors is OK **

Notification Type: RECOVERY Service: Puppet errors Host: deployment-mathoid Address: 10.68.23.236 State: OK Date/Time: Mon 03 Jul 20:21:40 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-pdfrender02/Puppet errors is OK **

Notification Type: RECOVERY Service: Puppet errors Host: deployment-pdfrender02 Address: 10.68.21.240 State: OK Date/Time: Mon 03 Jul 20:22:31 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-sca01/Puppet errors is OK **

Notification Type: RECOVERY Service: Puppet errors Host: deployment-sca01 Address: 10.68.20.183 State: OK Date/Time: Mon 03 Jul 20:25:38 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-sca03/Puppet errors is OK **

Notification Type: RECOVERY Service: Puppet errors Host: deployment-sca03 Address: 10.68.21.183 State: OK Date/Time: Mon 03 Jul 20:33:46 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-sca04/Puppet errors is OK **

Notification Type: RECOVERY Service: Puppet errors Host: deployment-sca04 Address: 10.68.18.80 State: OK Date/Time: Mon 03 Jul 20:35:40 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-fluorine02/Free space - all mounts is OK **

Notification Type: RECOVERY Service: Free space - all mounts Host: deployment-fluorine02 Address: 10.68.23.106 State: OK Date/Time: Mon 03 Jul 07:00:41 UTC 2017 Additional Info: OK: All targets OK ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-pdfrender02/Puppet errors is OK **

Notification Type: RECOVERY Service: Puppet errors Host: deployment-pdfrender02 Address: 10.68.21.240 State: OK Date/Time: Mon 03 Jul 06:23:20 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-fluorine02/Free space - all mounts is CRITICAL **

Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-fluorine02 Address: 10.68.23.106 State: CRITICAL Date/Time: Mon 03 Jul 06:35:38 UTC 2017 Additional Info: CRITICAL: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree (<11.11%)

[Betacluster-alerts] ** RECOVERY alert - deployment-urldownloader/Puppet errors is OK **

Notification Type: RECOVERY Service: Puppet errors Host: deployment-urldownloader Address: 10.68.16.135 State: OK Date/Time: Mon 03 Jul 13:02:57 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing

[Betacluster-alerts] ** PROBLEM alert - deployment-urldownloader/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-urldownloader Address: 10.68.16.135 State: CRITICAL Date/Time: Mon 03 Jul 12:22:57 UTC 2017 Additional Info: CRITICAL: 20.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-etcd-01/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-etcd-01 Address: 10.68.19.227 State: CRITICAL Date/Time: Mon 03 Jul 12:44:42 UTC 2017 Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-conf03/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-conf03 Address: 10.68.20.134 State: CRITICAL Date/Time: Mon 03 Jul 13:04:46 UTC 2017 Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-pdf01/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-pdf01 Address: 10.68.16.73 State: CRITICAL Date/Time: Mon 03 Jul 10:59:31 UTC 2017 Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** PROBLEM alert - deployment-mira/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-mira Address: 10.68.20.135 State: CRITICAL Date/Time: Mon 03 Jul 15:52:43 UTC 2017 Additional Info: CRITICAL: 22.22% of data above the critical threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** PROBLEM alert - deployment-eventlogging03/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-eventlogging03 Address: 10.68.18.111 State: CRITICAL Date/Time: Mon 03 Jul 15:27:30 UTC 2017 Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is WARNING **

Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: WARNING Date/Time: Mon 03 Jul 15:28:10 UTC 2017 Additional Info: WARNING: 60.00% of data above the warning threshold [1.0] ___

[Betacluster-alerts] ** RECOVERY alert - Graphite Labs/Mediawiki Error Rate is OK **

Notification Type: RECOVERY Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: OK Date/Time: Mon 03 Jul 15:33:10 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [1.0] ___ Betacluster-alerts

[Betacluster-alerts] ** PROBLEM alert - deployment-prometheus01/Puppet staleness is CRITICAL **

Notification Type: PROBLEM Service: Puppet staleness Host: deployment-prometheus01 Address: 10.68.20.247 State: CRITICAL Date/Time: Mon 03 Jul 13:26:53 UTC 2017 Additional Info: CRITICAL: 100.00% of data above the critical threshold [43200.0] ___

[Betacluster-alerts] beta-scap-eqiad - Build # 162402 - Failure!

beta-scap-eqiad - Build # 162402 - Failure: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/162402/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** PROBLEM alert - deployment-imagescaler01/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-imagescaler01 Address: 10.68.19.158 State: CRITICAL Date/Time: Mon 03 Jul 19:31:55 UTC 2017 Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] beta-scap-eqiad - Build # 162403 - Still Failing!

beta-scap-eqiad - Build # 162403 - Still Failing: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/162403/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] beta-scap-eqiad - Build # 162404 - Still Failing!

beta-scap-eqiad - Build # 162404 - Still Failing: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/162404/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** PROBLEM alert - deployment-mathoid/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-mathoid Address: 10.68.23.236 State: CRITICAL Date/Time: Mon 03 Jul 19:46:40 UTC 2017 Additional Info: CRITICAL: 66.67% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-pdfrender02/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-pdfrender02 Address: 10.68.21.240 State: CRITICAL Date/Time: Mon 03 Jul 19:47:32 UTC 2017 Additional Info: CRITICAL: 55.56% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-sca01/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-sca01 Address: 10.68.20.183 State: CRITICAL Date/Time: Mon 03 Jul 19:55:37 UTC 2017 Additional Info: CRITICAL: 33.33% of data above the critical threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] beta-scap-eqiad - Build # 162405 - Still Failing!

beta-scap-eqiad - Build # 162405 - Still Failing: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/162405/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** PROBLEM alert - deployment-sca03/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-sca03 Address: 10.68.21.183 State: CRITICAL Date/Time: Mon 03 Jul 19:58:47 UTC 2017 Additional Info: CRITICAL: 30.00% of data above the critical threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is WARNING **

Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: WARNING Date/Time: Mon 03 Jul 16:24:09 UTC 2017 Additional Info: WARNING: 20.00% of data above the warning threshold [1.0] ___

[Betacluster-alerts] ** RECOVERY alert - Graphite Labs/Mediawiki Error Rate is OK **

Notification Type: RECOVERY Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: OK Date/Time: Mon 03 Jul 16:34:10 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [1.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - Graphite Labs/Mediawiki Error Rate is OK **

Notification Type: RECOVERY Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: OK Date/Time: Mon 03 Jul 17:35:12 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [1.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - deployment-mira/Puppet errors is OK **

Notification Type: RECOVERY Service: Puppet errors Host: deployment-mira Address: 10.68.20.135 State: OK Date/Time: Mon 03 Jul 16:32:41 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is WARNING **

Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: WARNING Date/Time: Mon 03 Jul 17:25:10 UTC 2017 Additional Info: WARNING: 40.00% of data above the warning threshold [1.0] ___