[Betacluster-alerts] ** PROBLEM alert - deployment-puppetmaster02/Long lived cherry-picks on puppetmaster is CRITICAL **

Notification Type: PROBLEM Service: Long lived cherry-picks on puppetmaster Host: deployment-puppetmaster02 Address: 10.68.21.200 State: CRITICAL Date/Time: Wed 30 Nov 15:44:25 UTC 2016 Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0]

[Betacluster-alerts] beta-code-update-eqiad - Build # 132416 - Failure!

beta-code-update-eqiad - Build # 132416 - Failure: Check console output at https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/132416/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] beta-code-update-eqiad - Build # 132417 - Fixed!

beta-code-update-eqiad - Build # 132417 - Fixed: Check console output at https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/132417/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** RECOVERY alert - deployment-cache-upload04/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-cache-upload04 Address: 10.68.18.109 State: OK Date/Time: Wed 30 Nov 20:47:28 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-restbase02/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-restbase02 Address: 10.68.17.189 State: CRITICAL Date/Time: Wed 30 Nov 21:28:08 UTC 2016 Additional Info: CRITICAL: 22.22% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-ms-be01/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-ms-be01 Address: 10.68.16.24 State: CRITICAL Date/Time: Wed 30 Nov 21:34:06 UTC 2016 Additional Info: CRITICAL: 66.67% of data above the critical threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** PROBLEM alert - deployment-kafka03/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-kafka03 Address: 10.68.16.138 State: CRITICAL Date/Time: Wed 30 Nov 21:34:13 UTC 2016 Additional Info: CRITICAL: 66.67% of data above the critical threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - deployment-sca02/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-sca02 Address: 10.68.20.153 State: OK Date/Time: Wed 30 Nov 22:05:31 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-stream/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-stream Address: 10.68.17.106 State: OK Date/Time: Wed 30 Nov 22:05:29 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-poolcounter04/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-poolcounter04 Address: 10.68.17.48 State: OK Date/Time: Wed 30 Nov 22:05:32 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-ms-be01/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-ms-be01 Address: 10.68.16.24 State: OK Date/Time: Wed 30 Nov 22:09:02 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-kafka03/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-kafka03 Address: 10.68.16.138 State: OK Date/Time: Wed 30 Nov 22:09:12 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-cache-upload04/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-cache-upload04 Address: 10.68.18.109 State: CRITICAL Date/Time: Wed 30 Nov 20:07:30 UTC 2016 Additional Info: CRITICAL: 40.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** RECOVERY alert - deployment-ores-redis/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-ores-redis Address: 10.68.21.235 State: OK Date/Time: Wed 30 Nov 22:00:02 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-memc04/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-memc04 Address: 10.68.23.25 State: OK Date/Time: Wed 30 Nov 22:08:12 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-sca01/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-sca01 Address: 10.68.20.183 State: OK Date/Time: Wed 30 Nov 22:08:15 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-restbase02/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-restbase02 Address: 10.68.17.189 State: OK Date/Time: Wed 30 Nov 22:08:06 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-secureredirexperiment/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-secureredirexperiment Address: 10.68.17.132 State: OK Date/Time: Wed 30 Nov 22:07:24 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - deployment-sentry01/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-sentry01 Address: 10.68.19.148 State: OK Date/Time: Wed 30 Nov 22:07:16 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-tin/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-tin Address: 10.68.21.205 State: OK Date/Time: Wed 30 Nov 22:07:26 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-pdf01/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-pdf01 Address: 10.68.16.73 State: OK Date/Time: Wed 30 Nov 22:07:35 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-ores-redis/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-ores-redis Address: 10.68.21.235 State: CRITICAL Date/Time: Wed 30 Nov 21:25:03 UTC 2016 Additional Info: CRITICAL: 44.44% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-puppetmaster02/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-puppetmaster02 Address: 10.68.21.200 State: CRITICAL Date/Time: Wed 30 Nov 21:24:24 UTC 2016 Additional Info: CRITICAL: 44.44% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** RECOVERY alert - deployment-ircd/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-ircd Address: 10.68.20.19 State: OK Date/Time: Wed 30 Nov 22:07:09 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-kafka01/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-kafka01 Address: 10.68.21.219 State: OK Date/Time: Wed 30 Nov 22:12:04 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-pdf01/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-pdf01 Address: 10.68.16.73 State: CRITICAL Date/Time: Wed 30 Nov 21:27:35 UTC 2016 Additional Info: CRITICAL: 30.00% of data above the critical threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** PROBLEM alert - deployment-tin/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-tin Address: 10.68.21.205 State: CRITICAL Date/Time: Wed 30 Nov 21:27:26 UTC 2016 Additional Info: CRITICAL: 33.33% of data above the critical threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** PROBLEM alert - deployment-kafka05/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-kafka05 Address: 10.68.21.106 State: CRITICAL Date/Time: Wed 30 Nov 21:27:03 UTC 2016 Additional Info: CRITICAL: 66.67% of data above the critical threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - deployment-eventlogging03/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-eventlogging03 Address: 10.68.18.111 State: OK Date/Time: Wed 30 Nov 22:02:17 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-mx/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-mx Address: 10.68.17.78 State: OK Date/Time: Wed 30 Nov 22:02:46 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-db03/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-db03 Address: 10.68.23.30 State: OK Date/Time: Wed 30 Nov 22:13:33 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-mediawiki04/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-mediawiki04 Address: 10.68.19.128 State: CRITICAL Date/Time: Wed 30 Nov 21:23:12 UTC 2016 Additional Info: CRITICAL: 22.22% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-mx/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-mx Address: 10.68.17.78 State: CRITICAL Date/Time: Wed 30 Nov 21:22:46 UTC 2016 Additional Info: CRITICAL: 20.00% of data above the critical threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** PROBLEM alert - deployment-eventlogging03/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-eventlogging03 Address: 10.68.18.111 State: CRITICAL Date/Time: Wed 30 Nov 21:22:16 UTC 2016 Additional Info: CRITICAL: 22.22% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-db03/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-db03 Address: 10.68.23.30 State: CRITICAL Date/Time: Wed 30 Nov 21:33:35 UTC 2016 Additional Info: CRITICAL: 33.33% of data above the critical threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - deployment-imagescaler01/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-imagescaler01 Address: 10.68.19.158 State: OK Date/Time: Wed 30 Nov 22:10:10 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-aqs01/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-aqs01 Address: 10.68.18.237 State: CRITICAL Date/Time: Wed 30 Nov 21:26:04 UTC 2016 Additional Info: CRITICAL: 44.44% of data above the critical threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** PROBLEM alert - deployment-sca02/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-sca02 Address: 10.68.20.153 State: CRITICAL Date/Time: Wed 30 Nov 21:30:29 UTC 2016 Additional Info: CRITICAL: 55.56% of data above the critical threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** PROBLEM alert - deployment-poolcounter04/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-poolcounter04 Address: 10.68.17.48 State: CRITICAL Date/Time: Wed 30 Nov 21:30:33 UTC 2016 Additional Info: CRITICAL: 50.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-imagescaler01/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-imagescaler01 Address: 10.68.19.158 State: CRITICAL Date/Time: Wed 30 Nov 21:30:09 UTC 2016 Additional Info: CRITICAL: 22.22% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-ircd/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-ircd Address: 10.68.20.19 State: CRITICAL Date/Time: Wed 30 Nov 21:32:09 UTC 2016 Additional Info: CRITICAL: 66.67% of data above the critical threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** PROBLEM alert - deployment-memc04/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-memc04 Address: 10.68.23.25 State: CRITICAL Date/Time: Wed 30 Nov 21:33:12 UTC 2016 Additional Info: CRITICAL: 55.56% of data above the critical threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** PROBLEM alert - deployment-sentry01/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-sentry01 Address: 10.68.19.148 State: CRITICAL Date/Time: Wed 30 Nov 21:32:16 UTC 2016 Additional Info: CRITICAL: 66.67% of data above the critical threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** PROBLEM alert - deployment-kafka01/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-kafka01 Address: 10.68.21.219 State: CRITICAL Date/Time: Wed 30 Nov 21:32:04 UTC 2016 Additional Info: CRITICAL: 22.22% of data above the critical threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - deployment-mediawiki04/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-mediawiki04 Address: 10.68.19.128 State: OK Date/Time: Wed 30 Nov 22:03:13 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list