[Betacluster-alerts] ** PROBLEM alert - deployment-kafka03/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-kafka03 Address: 10.68.16.138 State: CRITICAL Date/Time: Tue 16 Jan 20:18:05 UTC 2018 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-pdfrender02/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-pdfrender02 Address: 10.68.21.240 State: CRITICAL Date/Time: Tue 16 Jan 20:19:11 UTC 2018 Notes URLs: Additional Info: CRITICAL: 20.00% of data above the critical threshold [0.0]

[Betacluster-alerts] ** RECOVERY alert - Generic Beta Cluster/English Wikipedia Main page is OK **

Notification Type: RECOVERY Service: English Wikipedia Main page Host: Generic Beta Cluster Address: en.wikipedia.beta.wmflabs.org State: OK Date/Time: Tue 16 Jan 20:42:08 UTC 2018 Notes URLs: Additional Info: HTTP OK: HTTP/1.1 200 OK - 47380 bytes in 1.242 second response time

[Betacluster-alerts] ** PROBLEM alert - deployment-memc06/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-memc06 Address: 10.68.22.239 State: CRITICAL Date/Time: Tue 16 Jan 20:42:31 UTC 2018 Notes URLs: Additional Info: CRITICAL: 50.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-aqs01/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-aqs01 Address: 10.68.18.237 State: CRITICAL Date/Time: Tue 16 Jan 20:42:55 UTC 2018 Notes URLs: Additional Info: CRITICAL: 66.67% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-imagescaler02/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-imagescaler02 Address: 10.68.18.233 State: CRITICAL Date/Time: Tue 16 Jan 20:43:22 UTC 2018 Notes URLs: Additional Info: CRITICAL: 44.44% of data above the critical threshold [0.0]

[Betacluster-alerts] ** RECOVERY alert - deployment-mediawiki05/SSH is OK **

Notification Type: RECOVERY Service: SSH Host: deployment-mediawiki05 Address: 10.68.22.21 State: OK Date/Time: Tue 16 Jan 20:59:19 UTC 2018 Notes URLs: Additional Info: SSH OK - OpenSSH_6.7p1 Debian-5+deb8u3 (protocol 2.0) ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - deployment-tin/SSH is OK **

Notification Type: RECOVERY Service: SSH Host: deployment-tin Address: 10.68.21.205 State: OK Date/Time: Tue 16 Jan 20:59:24 UTC 2018 Notes URLs: Additional Info: SSH OK - OpenSSH_6.7p1 Debian-5+deb8u4 (protocol 2.0) ___ Betacluster-alerts mailing

[Betacluster-alerts] Host UP alert for deployment-snapshot01!

Notification Type: RECOVERY Host: deployment-snapshot01 State: UP Address: 10.68.19.94 Info: PING OK - Packet loss = 0%, RTA = 0.77 ms Date/Time: Tue 16 Jan 21:22:54 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** RECOVERY alert - deployment-kafka-jumbo-1/Puppet errors is OK **

Notification Type: RECOVERY Service: Puppet errors Host: deployment-kafka-jumbo-1 Address: 10.68.23.243 State: OK Date/Time: Tue 16 Jan 21:22:20 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [0.0] ___

[Betacluster-alerts] Host UP alert for deployment-ores01!

Notification Type: RECOVERY Host: deployment-ores01 State: UP Address: 10.68.23.165 Info: PING OK - Packet loss = 0%, RTA = 1.32 ms Date/Time: Tue 16 Jan 21:22:40 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] Host UP alert for deployment-dumps-puppetmaster!

Notification Type: RECOVERY Host: deployment-dumps-puppetmaster State: UP Address: 10.68.21.153 Info: PING OK - Packet loss = 0%, RTA = 0.88 ms Date/Time: Tue 16 Jan 21:22:13 UTC 2018 ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-mediawiki07/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-mediawiki07 Address: 10.68.17.40 State: CRITICAL Date/Time: Tue 16 Jan 21:34:01 UTC 2018 Notes URLs: Additional Info: CRITICAL: 16.67% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-tin/Free space - all mounts is WARNING **

Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-tin Address: 10.68.21.205 State: WARNING Date/Time: Tue 16 Jan 21:59:40 UTC 2018 Notes URLs: Additional Info: WARNING: deployment-prep.deployment-tin.diskspace._mnt.byte_percentfree (No valid datapoints

[Betacluster-alerts] ** PROBLEM alert - deployment-zotero01/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-zotero01 Address: 10.68.17.102 State: CRITICAL Date/Time: Tue 16 Jan 21:59:16 UTC 2018 Notes URLs: Additional Info: CRITICAL: 33.33% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** RECOVERY alert - deployment-kafka03/Puppet staleness is OK **

Notification Type: RECOVERY Service: Puppet staleness Host: deployment-kafka03 Address: 10.68.16.138 State: OK Date/Time: Tue 16 Jan 20:19:33 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [3600.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-kafka01/Free space - all mounts is CRITICAL **

Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-kafka01 Address: 10.68.21.219 State: CRITICAL Date/Time: Tue 16 Jan 20:19:43 UTC 2018 Notes URLs: Additional Info: CRITICAL: deployment-prep.deployment-kafka01.diskspace.root.byte_percentfree (<100.00%)

[Betacluster-alerts] Host UP alert for deployment-cache-upload04!

Notification Type: RECOVERY Host: deployment-cache-upload04 State: UP Address: 10.68.18.109 Info: PING OK - Packet loss = 0%, RTA = 2.94 ms Date/Time: Tue 16 Jan 20:27:49 UTC 2018 ___ Betacluster-alerts mailing list

[Betacluster-alerts] Host UP alert for deployment-puppetmaster02!

Notification Type: RECOVERY Host: deployment-puppetmaster02 State: UP Address: 10.68.21.200 Info: PING OK - Packet loss = 0%, RTA = 0.64 ms Date/Time: Tue 16 Jan 20:41:41 UTC 2018 ___ Betacluster-alerts mailing list

[Betacluster-alerts] Host UP alert for deployment-mediawiki04!

Notification Type: RECOVERY Host: deployment-mediawiki04 State: UP Address: 10.68.19.128 Info: PING OK - Packet loss = 0%, RTA = 1.32 ms Date/Time: Tue 16 Jan 20:41:25 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] Host UP alert for deployment-sca04!

Notification Type: RECOVERY Host: deployment-sca04 State: UP Address: 10.68.18.80 Info: PING OK - Packet loss = 0%, RTA = 0.50 ms Date/Time: Tue 16 Jan 20:41:43 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] Host UP alert for deployment-mira!

Notification Type: RECOVERY Host: deployment-mira State: UP Address: 10.68.20.135 Info: PING OK - Packet loss = 0%, RTA = 0.56 ms Date/Time: Tue 16 Jan 20:41:00 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** RECOVERY alert - Generic Beta Cluster/English Wikipedia Mobile Main page is OK **

Notification Type: RECOVERY Service: English Wikipedia Mobile Main page Host: Generic Beta Cluster Address: en.wikipedia.beta.wmflabs.org State: OK Date/Time: Tue 16 Jan 20:41:45 UTC 2018 Notes URLs: Additional Info: HTTP OK: HTTP/1.1 200 OK - 35900 bytes in 1.570 second response time

[Betacluster-alerts] Host UP alert for deployment-parsoid09!

Notification Type: RECOVERY Host: deployment-parsoid09 State: UP Address: 10.68.20.142 Info: PING OK - Packet loss = 0%, RTA = 0.94 ms Date/Time: Tue 16 Jan 20:40:58 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** PROBLEM alert - deployment-elastic05/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-elastic05 Address: 10.68.20.21 State: CRITICAL Date/Time: Tue 16 Jan 20:41:12 UTC 2018 Notes URLs: Additional Info: CRITICAL: 44.44% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-sca01/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-sca01 Address: 10.68.20.183 State: CRITICAL Date/Time: Tue 16 Jan 20:40:50 UTC 2018 Notes URLs: Additional Info: CRITICAL: 33.33% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** RECOVERY alert - deployment-prometheus01/Puppet errors is OK **

Notification Type: RECOVERY Service: Puppet errors Host: deployment-prometheus01 Address: 10.68.20.247 State: OK Date/Time: Tue 16 Jan 21:06:54 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [0.0] ___

[Betacluster-alerts] beta-code-update-eqiad - Build # 189817 - Fixed!

beta-code-update-eqiad - Build # 189817 - Fixed: Check console output at https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/189817/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] beta-scap-eqiad - Build # 191129 - Failure!

beta-scap-eqiad - Build # 191129 - Failure: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/191129/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** RECOVERY alert - deployment-changeprop/Puppet errors is OK **

Notification Type: RECOVERY Service: Puppet errors Host: deployment-changeprop Address: 10.68.16.88 State: OK Date/Time: Tue 16 Jan 21:17:44 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - deployment-aqs01/Puppet errors is OK **

Notification Type: RECOVERY Service: Puppet errors Host: deployment-aqs01 Address: 10.68.18.237 State: OK Date/Time: Tue 16 Jan 21:17:55 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] Host DOWN alert for deployment-snapshot01!

Notification Type: PROBLEM Host: deployment-snapshot01 State: DOWN Address: 10.68.19.94 Info: CRITICAL - Host Unreachable (10.68.19.94) Date/Time: Tue 16 Jan 21:17:29 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] Host DOWN alert for deployment-mx02!

Notification Type: PROBLEM Host: deployment-mx02 State: DOWN Address: 10.68.23.220 Info: PING CRITICAL - Packet loss = 100% Date/Time: Tue 16 Jan 21:16:32 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] Host DOWN alert for deployment-memc07!

Notification Type: PROBLEM Host: deployment-memc07 State: DOWN Address: 10.68.17.171 Info: CRITICAL - Host Unreachable (10.68.17.171) Date/Time: Tue 16 Jan 21:16:42 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] Host UP alert for deployment-imagescaler02!

Notification Type: RECOVERY Host: deployment-imagescaler02 State: UP Address: 10.68.18.233 Info: PING OK - Packet loss = 0%, RTA = 1.55 ms Date/Time: Tue 16 Jan 21:28:24 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] Host UP alert for deployment-kafka-jumbo-2!

Notification Type: RECOVERY Host: deployment-kafka-jumbo-2 State: UP Address: 10.68.16.87 Info: PING OK - Packet loss = 0%, RTA = 0.79 ms Date/Time: Tue 16 Jan 21:28:37 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] Host UP alert for deployment-memc06!

Notification Type: RECOVERY Host: deployment-memc06 State: UP Address: 10.68.22.239 Info: PING OK - Packet loss = 0%, RTA = 3.69 ms Date/Time: Tue 16 Jan 21:30:21 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** RECOVERY alert - deployment-zotero01/Puppet errors is OK **

Notification Type: RECOVERY Service: Puppet errors Host: deployment-zotero01 Address: 10.68.17.102 State: OK Date/Time: Tue 16 Jan 22:34:16 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** PROBLEM alert - deployment-mediawiki05/App Server Main HTTP Response is CRITICAL **

Notification Type: PROBLEM Service: App Server Main HTTP Response Host: deployment-mediawiki05 Address: 10.68.22.21 State: CRITICAL Date/Time: Tue 16 Jan 22:45:03 UTC 2018 Notes URLs: Additional Info: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - string 'Wikipedia' not found on

[Betacluster-alerts] beta-scap-eqiad - Build # 191127 - Failure!

beta-scap-eqiad - Build # 191127 - Failure: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/191127/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** PROBLEM alert - Generic Beta Cluster/English Wikipedia Main page is CRITICAL **

Notification Type: PROBLEM Service: English Wikipedia Main page Host: Generic Beta Cluster Address: en.wikipedia.beta.wmflabs.org State: CRITICAL Date/Time: Tue 16 Jan 20:37:09 UTC 2018 Notes URLs: Additional Info: HTTP CRITICAL: HTTP/1.1 503 Backend fetch failed - string 'Wikipedia' not

[Betacluster-alerts] ** PROBLEM alert - Generic Beta Cluster/English Wikipedia Mobile Main page is CRITICAL **

Notification Type: PROBLEM Service: English Wikipedia Mobile Main page Host: Generic Beta Cluster Address: en.wikipedia.beta.wmflabs.org State: CRITICAL Date/Time: Tue 16 Jan 20:36:46 UTC 2018 Notes URLs: Additional Info: HTTP CRITICAL: HTTP/1.1 503 Backend fetch failed - string 'Wikipedia'

[Betacluster-alerts] Host DOWN alert for deployment-sca04!

Notification Type: PROBLEM Host: deployment-sca04 State: DOWN Address: 10.68.18.80 Info: CRITICAL - Host Unreachable (10.68.18.80) Date/Time: Tue 16 Jan 20:36:34 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] Host DOWN alert for deployment-mediawiki04!

Notification Type: PROBLEM Host: deployment-mediawiki04 State: DOWN Address: 10.68.19.128 Info: CRITICAL - Host Unreachable (10.68.19.128) Date/Time: Tue 16 Jan 20:37:29 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** PROBLEM alert - deployment-urldownloader/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-urldownloader Address: 10.68.16.135 State: CRITICAL Date/Time: Tue 16 Jan 20:43:39 UTC 2018 Notes URLs: Additional Info: CRITICAL: 22.22% of data above the critical threshold [0.0]

[Betacluster-alerts] ** PROBLEM alert - deployment-mediawiki04/App Server Main HTTP Response is CRITICAL **

Notification Type: PROBLEM Service: App Server Main HTTP Response Host: deployment-mediawiki04 Address: 10.68.19.128 State: CRITICAL Date/Time: Tue 16 Jan 21:03:22 UTC 2018 Notes URLs: Additional Info: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 1603 bytes in 2.074 second response

[Betacluster-alerts] ** PROBLEM alert - Generic Beta Cluster/English Wikipedia Mobile Main page is CRITICAL **

Notification Type: PROBLEM Service: English Wikipedia Mobile Main page Host: Generic Beta Cluster Address: en.wikipedia.beta.wmflabs.org State: CRITICAL Date/Time: Tue 16 Jan 21:02:45 UTC 2018 Notes URLs: Additional Info: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 2203 bytes in

[Betacluster-alerts] ** PROBLEM alert - Generic Beta Cluster/English Wikipedia Main page is CRITICAL **

Notification Type: PROBLEM Service: English Wikipedia Main page Host: Generic Beta Cluster Address: en.wikipedia.beta.wmflabs.org State: CRITICAL Date/Time: Tue 16 Jan 21:03:08 UTC 2018 Notes URLs: Additional Info: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 2185 bytes in 1.121

[Betacluster-alerts] Host UP alert for deployment-urldownloader!

Notification Type: RECOVERY Host: deployment-urldownloader State: UP Address: 10.68.16.135 Info: PING OK - Packet loss = 0%, RTA = 1.43 ms Date/Time: Tue 16 Jan 21:07:33 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** PROBLEM alert - deployment-mx02/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-mx02 Address: 10.68.23.220 State: CRITICAL Date/Time: Tue 16 Jan 21:08:21 UTC 2018 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] Host UP alert for deployment-db04!

Notification Type: RECOVERY Host: deployment-db04 State: UP Address: 10.68.18.35 Info: PING OK - Packet loss = 0%, RTA = 0.81 ms Date/Time: Tue 16 Jan 21:08:46 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** RECOVERY alert - deployment-tin/Puppet errors is OK **

Notification Type: RECOVERY Service: Puppet errors Host: deployment-tin Address: 10.68.21.205 State: OK Date/Time: Tue 16 Jan 21:07:36 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - Generic Beta Cluster/English Wikipedia Mobile Main page is OK **

Notification Type: RECOVERY Service: English Wikipedia Mobile Main page Host: Generic Beta Cluster Address: en.wikipedia.beta.wmflabs.org State: OK Date/Time: Tue 16 Jan 21:07:45 UTC 2018 Notes URLs: Additional Info: HTTP OK: HTTP/1.1 200 OK - 35925 bytes in 2.376 second response time

[Betacluster-alerts] Host UP alert for deployment-mediawiki06!

Notification Type: RECOVERY Host: deployment-mediawiki06 State: UP Address: 10.68.19.241 Info: PING OK - Packet loss = 0%, RTA = 1.40 ms Date/Time: Tue 16 Jan 21:07:41 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] Host UP alert for deployment-apertium02!

Notification Type: RECOVERY Host: deployment-apertium02 State: UP Address: 10.68.22.254 Info: PING OK - Packet loss = 0%, RTA = 1.17 ms Date/Time: Tue 16 Jan 21:08:09 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] Host UP alert for deployment-mcs01!

Notification Type: RECOVERY Host: deployment-mcs01 State: UP Address: 10.68.17.18 Info: PING OK - Packet loss = 0%, RTA = 1.03 ms Date/Time: Tue 16 Jan 21:08:36 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** RECOVERY alert - Generic Beta Cluster/English Wikipedia Main page is OK **

Notification Type: RECOVERY Service: English Wikipedia Main page Host: Generic Beta Cluster Address: en.wikipedia.beta.wmflabs.org State: OK Date/Time: Tue 16 Jan 21:08:09 UTC 2018 Notes URLs: Additional Info: HTTP OK: HTTP/1.1 200 OK - 47378 bytes in 0.780 second response time

[Betacluster-alerts] ** RECOVERY alert - deployment-mediawiki04/App Server Main HTTP Response is OK **

Notification Type: RECOVERY Service: App Server Main HTTP Response Host: deployment-mediawiki04 Address: 10.68.19.128 State: OK Date/Time: Tue 16 Jan 21:08:22 UTC 2018 Notes URLs: Additional Info: HTTP OK: HTTP/1.1 200 OK - 46847 bytes in 0.841 second response time

[Betacluster-alerts] ** RECOVERY alert - deployment-sca01/Puppet errors is OK **

Notification Type: RECOVERY Service: Puppet errors Host: deployment-sca01 Address: 10.68.20.183 State: OK Date/Time: Tue 16 Jan 21:20:50 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - deployment-db04/Puppet errors is OK **

Notification Type: RECOVERY Service: Puppet errors Host: deployment-db04 Address: 10.68.18.35 State: OK Date/Time: Tue 16 Jan 21:20:04 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** PROBLEM alert - deployment-tin/Free space - all mounts is WARNING **

Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-tin Address: 10.68.21.205 State: WARNING Date/Time: Tue 16 Jan 22:30:38 UTC 2018 Notes URLs: Additional Info: WARNING: deployment-prep.deployment-tin.diskspace._mnt.byte_percentfree (No valid datapoints

[Betacluster-alerts] ** RECOVERY alert - deployment-tin/Free space - all mounts is OK **

Notification Type: RECOVERY Service: Free space - all mounts Host: deployment-tin Address: 10.68.21.205 State: OK Date/Time: Tue 16 Jan 20:32:41 UTC 2018 Notes URLs: Additional Info: OK: deployment-prep.deployment-tin.diskspace._mnt.byte_percentfree (No valid datapoints found)

[Betacluster-alerts] Host UP alert for deployment-mediawiki05!

Notification Type: RECOVERY Host: deployment-mediawiki05 State: UP Address: 10.68.22.21 Info: PING OK - Packet loss = 0%, RTA = 0.69 ms Date/Time: Tue 16 Jan 20:54:34 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** PROBLEM alert - deployment-mediawiki07/App Server Main HTTP Response is CRITICAL **

Notification Type: PROBLEM Service: App Server Main HTTP Response Host: deployment-mediawiki07 Address: 10.68.17.40 State: CRITICAL Date/Time: Tue 16 Jan 21:04:35 UTC 2018 Notes URLs: Additional Info: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 1598 bytes in 2.109 second response

[Betacluster-alerts] ** RECOVERY alert - deployment-mediawiki05/Puppet errors is OK **

Notification Type: RECOVERY Service: Puppet errors Host: deployment-mediawiki05 Address: 10.68.22.21 State: OK Date/Time: Tue 16 Jan 21:04:42 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] Host DOWN alert for deployment-kafka05!

Notification Type: PROBLEM Host: deployment-kafka05 State: DOWN Address: 10.68.21.106 Info: CRITICAL - Host Unreachable (10.68.21.106) Date/Time: Tue 16 Jan 21:04:06 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] Host DOWN alert for deployment-jobrunner02!

Notification Type: PROBLEM Host: deployment-jobrunner02 State: DOWN Address: 10.68.19.42 Info: CRITICAL - Host Unreachable (10.68.19.42) Date/Time: Tue 16 Jan 21:04:03 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] Host DOWN alert for deployment-ores01!

Notification Type: PROBLEM Host: deployment-ores01 State: DOWN Address: 10.68.23.165 Info: CRITICAL - Host Unreachable (10.68.23.165) Date/Time: Tue 16 Jan 21:19:27 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] Host DOWN alert for deployment-dumps-puppetmaster!

Notification Type: PROBLEM Host: deployment-dumps-puppetmaster State: DOWN Address: 10.68.21.153 Info: CRITICAL - Host Unreachable (10.68.21.153) Date/Time: Tue 16 Jan 21:19:43 UTC 2018 ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-mira/Puppet errors is OK **

Notification Type: RECOVERY Service: Puppet errors Host: deployment-mira Address: 10.68.20.135 State: OK Date/Time: Tue 16 Jan 21:20:04 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] beta-scap-eqiad - Build # 191133 - Fixed!

beta-scap-eqiad - Build # 191133 - Fixed: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/191133/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] Host UP alert for deployment-aqs03!

Notification Type: RECOVERY Host: deployment-aqs03 State: UP Address: 10.68.17.125 Info: PING OK - Packet loss = 0%, RTA = 2.56 ms Date/Time: Tue 16 Jan 20:27:25 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] Host DOWN alert for deployment-parsoid09!

Notification Type: PROBLEM Host: deployment-parsoid09 State: DOWN Address: 10.68.20.142 Info: CRITICAL - Host Unreachable (10.68.20.142) Date/Time: Tue 16 Jan 20:35:57 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** PROBLEM alert - deployment-imagescaler01/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-imagescaler01 Address: 10.68.19.158 State: CRITICAL Date/Time: Tue 16 Jan 20:46:39 UTC 2018 Notes URLs: Additional Info: CRITICAL: 60.00% of data above the critical threshold [0.0]

[Betacluster-alerts] ** PROBLEM alert - deployment-sca04/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-sca04 Address: 10.68.18.80 State: CRITICAL Date/Time: Tue 16 Jan 20:46:35 UTC 2018 Notes URLs: Additional Info: CRITICAL: 16.67% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** RECOVERY alert - deployment-sca04/Puppet errors is OK **

Notification Type: RECOVERY Service: Puppet errors Host: deployment-sca04 Address: 10.68.18.80 State: OK Date/Time: Tue 16 Jan 20:51:36 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] Host DOWN alert for deployment-tin!

Notification Type: PROBLEM Host: deployment-tin State: DOWN Address: 10.68.21.205 Info: CRITICAL - Host Unreachable (10.68.21.205) Date/Time: Tue 16 Jan 20:51:57 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] Host DOWN alert for deployment-apertium02!

Notification Type: PROBLEM Host: deployment-apertium02 State: DOWN Address: 10.68.22.254 Info: CRITICAL - Host Unreachable (10.68.22.254) Date/Time: Tue 16 Jan 21:00:55 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** PROBLEM alert - deployment-elastic07/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-elastic07 Address: 10.68.19.180 State: CRITICAL Date/Time: Tue 16 Jan 21:00:48 UTC 2018 Notes URLs: Additional Info: CRITICAL: 14.29% of data above the critical threshold [0.0] ___

[Betacluster-alerts] beta-scap-eqiad - Build # 191130 - Still Failing!

beta-scap-eqiad - Build # 191130 - Still Failing: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/191130/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] Host UP alert for deployment-kafka05!

Notification Type: RECOVERY Host: deployment-kafka05 State: UP Address: 10.68.21.106 Info: PING OK - Packet loss = 0%, RTA = 1.15 ms Date/Time: Tue 16 Jan 21:09:29 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** RECOVERY alert - deployment-mediawiki05/App Server Main HTTP Response is OK **

Notification Type: RECOVERY Service: App Server Main HTTP Response Host: deployment-mediawiki05 Address: 10.68.22.21 State: OK Date/Time: Tue 16 Jan 21:09:03 UTC 2018 Notes URLs: Additional Info: HTTP OK: HTTP/1.1 200 OK - 46833 bytes in 1.513 second response time

[Betacluster-alerts] ** RECOVERY alert - deployment-mediawiki07/App Server Main HTTP Response is OK **

Notification Type: RECOVERY Service: App Server Main HTTP Response Host: deployment-mediawiki07 Address: 10.68.17.40 State: OK Date/Time: Tue 16 Jan 21:09:35 UTC 2018 Notes URLs: Additional Info: HTTP OK: HTTP/1.1 200 OK - 46412 bytes in 0.988 second response time

[Betacluster-alerts] ** RECOVERY alert - deployment-elastic05/Puppet errors is OK **

Notification Type: RECOVERY Service: Puppet errors Host: deployment-elastic05 Address: 10.68.20.21 State: OK Date/Time: Tue 16 Jan 21:16:12 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] beta-scap-eqiad - Build # 191131 - Still Failing!

beta-scap-eqiad - Build # 191131 - Still Failing: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/191131/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] Host DOWN alert for deployment-mediawiki07!

Notification Type: PROBLEM Host: deployment-mediawiki07 State: DOWN Address: 10.68.17.40 Info: CRITICAL - Host Unreachable (10.68.17.40) Date/Time: Tue 16 Jan 21:15:41 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] Host DOWN alert for deployment-eventlog02!

Notification Type: PROBLEM Host: deployment-eventlog02 State: DOWN Address: 10.68.18.138 Info: CRITICAL - Host Unreachable (10.68.18.138) Date/Time: Tue 16 Jan 21:16:18 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** RECOVERY alert - deployment-kafka03/Free space - all mounts is OK **

Notification Type: RECOVERY Service: Free space - all mounts Host: deployment-kafka03 Address: 10.68.16.138 State: OK Date/Time: Tue 16 Jan 22:20:29 UTC 2018 Notes URLs: Additional Info: OK: All targets OK ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-pdfrender02/Puppet errors is OK **

Notification Type: RECOVERY Service: Puppet errors Host: deployment-pdfrender02 Address: 10.68.21.240 State: OK Date/Time: Tue 16 Jan 20:24:13 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-changeprop/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-changeprop Address: 10.68.16.88 State: CRITICAL Date/Time: Tue 16 Jan 20:37:44 UTC 2018 Notes URLs: Additional Info: CRITICAL: 30.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-ms-be04/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-ms-be04 Address: 10.68.16.139 State: CRITICAL Date/Time: Tue 16 Jan 20:38:40 UTC 2018 Notes URLs: Additional Info: CRITICAL: 40.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] Host DOWN alert for deployment-poolcounter04!

Notification Type: PROBLEM Host: deployment-poolcounter04 State: DOWN Address: 10.68.17.48 Info: CRITICAL - Host Unreachable (10.68.17.48) Date/Time: Tue 16 Jan 20:38:00 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] Host DOWN alert for deployment-mediawiki06!

Notification Type: PROBLEM Host: deployment-mediawiki06 State: DOWN Address: 10.68.19.241 Info: CRITICAL - Host Unreachable (10.68.19.241) Date/Time: Tue 16 Jan 21:02:36 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** PROBLEM alert - deployment-tin/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-tin Address: 10.68.21.205 State: CRITICAL Date/Time: Tue 16 Jan 21:02:38 UTC 2018 Notes URLs: Additional Info: CRITICAL: 12.50% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-prometheus01/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-prometheus01 Address: 10.68.20.247 State: CRITICAL Date/Time: Tue 16 Jan 21:01:55 UTC 2018 Notes URLs: Additional Info: CRITICAL: 12.50% of data above the critical threshold [0.0]

[Betacluster-alerts] Host DOWN alert for deployment-db04!

Notification Type: PROBLEM Host: deployment-db04 State: DOWN Address: 10.68.18.35 Info: CRITICAL - Host Unreachable (10.68.18.35) Date/Time: Tue 16 Jan 21:01:45 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] Host DOWN alert for deployment-urldownloader!

Notification Type: PROBLEM Host: deployment-urldownloader State: DOWN Address: 10.68.16.135 Info: CRITICAL - Host Unreachable (10.68.16.135) Date/Time: Tue 16 Jan 21:01:43 UTC 2018 ___ Betacluster-alerts mailing list

[Betacluster-alerts] Host DOWN alert for deployment-mcs01!

Notification Type: PROBLEM Host: deployment-mcs01 State: DOWN Address: 10.68.17.18 Info: CRITICAL - Host Unreachable (10.68.17.18) Date/Time: Tue 16 Jan 21:01:28 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] Host DOWN alert for deployment-imagescaler02!

Notification Type: PROBLEM Host: deployment-imagescaler02 State: DOWN Address: 10.68.18.233 Info: CRITICAL - Host Unreachable (10.68.18.233) Date/Time: Tue 16 Jan 21:13:36 UTC 2018 ___ Betacluster-alerts mailing list

[Betacluster-alerts] Host UP alert for deployment-mx02!

Notification Type: RECOVERY Host: deployment-mx02 State: UP Address: 10.68.23.220 Info: PING OK - Packet loss = 0%, RTA = 1.46 ms Date/Time: Tue 16 Jan 21:23:55 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

  1   2   3   >