[Betacluster-alerts] ** PROBLEM alert - Generic Beta Cluster/English Wikipedia Mobile Main page is CRITICAL **

2018-11-19 Thread shinken
Notification Type: PROBLEM Service: English Wikipedia Mobile Main page Host: Generic Beta Cluster Address: en.wikipedia.beta.wmflabs.org State: CRITICAL Date/Time: Tue 20 Nov 01:18:05 UTC 2018 Notes URLs: Additional Info: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 2783 bytes in

[Betacluster-alerts] Host DOWN alert for deployment-puppetmaster03!

2018-11-19 Thread shinken
Notification Type: PROBLEM Host: deployment-puppetmaster03 State: DOWN Address: 10.68.23.29 Info: CRITICAL - Host Unreachable (10.68.23.29) Date/Time: Tue 20 Nov 00:15:45 UTC 2018 ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is CRITICAL **

2018-11-19 Thread shinken
Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: CRITICAL Date/Time: Tue 20 Nov 00:15:20 UTC 2018 Notes URLs: Additional Info: CRITICAL: 40.00% of data above the critical threshold [10.0]

[Betacluster-alerts] ** PROBLEM alert - deployment-memc06/SSH is CRITICAL **

2018-11-19 Thread shinken
Notification Type: PROBLEM Service: SSH Host: deployment-memc06 Address: 10.68.22.239 State: CRITICAL Date/Time: Tue 20 Nov 00:15:13 UTC 2018 Notes URLs: Additional Info: CRITICAL - Socket timeout after 10 seconds ___ Betacluster-alerts mailing

[Betacluster-alerts] ** PROBLEM alert - deployment-ircd/SSH is CRITICAL **

2018-11-19 Thread shinken
Notification Type: PROBLEM Service: SSH Host: deployment-ircd Address: 10.68.20.19 State: CRITICAL Date/Time: Tue 20 Nov 00:15:40 UTC 2018 Notes URLs: Additional Info: CRITICAL - Socket timeout after 10 seconds ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-elastic05/SSH is CRITICAL **

2018-11-19 Thread shinken
Notification Type: PROBLEM Service: SSH Host: deployment-elastic05 Address: 10.68.18.29 State: CRITICAL Date/Time: Tue 20 Nov 00:15:42 UTC 2018 Notes URLs: Additional Info: CRITICAL - Socket timeout after 10 seconds ___ Betacluster-alerts mailing

[Betacluster-alerts] ** PROBLEM alert - deployment-etcd-01/SSH is CRITICAL **

2018-11-19 Thread shinken
Notification Type: PROBLEM Service: SSH Host: deployment-etcd-01 Address: 10.68.19.227 State: CRITICAL Date/Time: Tue 20 Nov 00:15:45 UTC 2018 Notes URLs: Additional Info: CRITICAL - Socket timeout after 10 seconds ___ Betacluster-alerts mailing

[Betacluster-alerts] ** RECOVERY alert - Graphite Labs/Mediawiki Error Rate is OK **

2018-11-19 Thread shinken
Notification Type: RECOVERY Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: OK Date/Time: Tue 20 Nov 00:35:22 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [1.0] ___

[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is CRITICAL **

2018-11-19 Thread shinken
Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: CRITICAL Date/Time: Tue 20 Nov 01:21:21 UTC 2018 Notes URLs: Additional Info: CRITICAL: 20.00% of data above the critical threshold [10.0]

[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is WARNING **

2018-11-19 Thread shinken
Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: WARNING Date/Time: Tue 20 Nov 01:41:21 UTC 2018 Notes URLs: Additional Info: WARNING: 100.00% of data above the warning threshold [1.0]

[Betacluster-alerts] ** PROBLEM alert - deployment-memc05/SSH is CRITICAL **

2018-11-19 Thread shinken
Notification Type: PROBLEM Service: SSH Host: deployment-memc05 Address: 10.68.23.49 State: CRITICAL Date/Time: Tue 20 Nov 00:18:00 UTC 2018 Notes URLs: Additional Info: CRITICAL - Socket timeout after 10 seconds ___ Betacluster-alerts mailing

[Betacluster-alerts] ** PROBLEM alert - deployment-kafka-jumbo-2/SSH is CRITICAL **

2018-11-19 Thread shinken
Notification Type: PROBLEM Service: SSH Host: deployment-kafka-jumbo-2 Address: 10.68.16.87 State: CRITICAL Date/Time: Tue 20 Nov 00:17:49 UTC 2018 Notes URLs: Additional Info: CRITICAL - Socket timeout after 10 seconds ___ Betacluster-alerts

[Betacluster-alerts] ** PROBLEM alert - deployment-db03/SSH is CRITICAL **

2018-11-19 Thread shinken
Notification Type: PROBLEM Service: SSH Host: deployment-db03 Address: 10.68.23.30 State: CRITICAL Date/Time: Tue 20 Nov 00:17:28 UTC 2018 Notes URLs: Additional Info: CRITICAL - Socket timeout after 10 seconds ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-cache-upload04/SSH is CRITICAL **

2018-11-19 Thread shinken
Notification Type: PROBLEM Service: SSH Host: deployment-cache-upload04 Address: 10.68.18.109 State: CRITICAL Date/Time: Tue 20 Nov 00:17:58 UTC 2018 Notes URLs: Additional Info: CRITICAL - Socket timeout after 10 seconds ___ Betacluster-alerts

[Betacluster-alerts] ** PROBLEM alert - deployment-mathoid/SSH is CRITICAL **

2018-11-19 Thread shinken
Notification Type: PROBLEM Service: SSH Host: deployment-mathoid Address: 10.68.23.236 State: CRITICAL Date/Time: Tue 20 Nov 00:17:51 UTC 2018 Notes URLs: Additional Info: CRITICAL - Socket timeout after 10 seconds ___ Betacluster-alerts mailing

[Betacluster-alerts] ** PROBLEM alert - deployment-changeprop/SSH is CRITICAL **

2018-11-19 Thread shinken
Notification Type: PROBLEM Service: SSH Host: deployment-changeprop Address: 10.68.16.88 State: CRITICAL Date/Time: Tue 20 Nov 00:17:09 UTC 2018 Notes URLs: Additional Info: CRITICAL - Socket timeout after 10 seconds ___ Betacluster-alerts mailing

[Betacluster-alerts] Host DOWN alert for deployment-dumps-puppetmaster02!

2018-11-19 Thread shinken
Notification Type: PROBLEM Host: deployment-dumps-puppetmaster02 State: DOWN Address: 10.68.20.216 Info: CRITICAL - Host Unreachable (10.68.20.216) Date/Time: Tue 20 Nov 00:17:35 UTC 2018 ___ Betacluster-alerts mailing list

[Betacluster-alerts] Host DOWN alert for deployment-memc07!

2018-11-19 Thread shinken
Notification Type: PROBLEM Host: deployment-memc07 State: DOWN Address: 10.68.17.171 Info: CRITICAL - Host Unreachable (10.68.17.171) Date/Time: Tue 20 Nov 00:17:21 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** PROBLEM alert - deployment-db04/SSH is CRITICAL **

2018-11-19 Thread shinken
Notification Type: PROBLEM Service: SSH Host: deployment-db04 Address: 10.68.18.35 State: CRITICAL Date/Time: Tue 20 Nov 00:17:24 UTC 2018 Notes URLs: Additional Info: CRITICAL - Socket timeout after 10 seconds ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-mcs01/SSH is CRITICAL **

2018-11-19 Thread shinken
Notification Type: PROBLEM Service: SSH Host: deployment-mcs01 Address: 10.68.17.18 State: CRITICAL Date/Time: Tue 20 Nov 00:17:28 UTC 2018 Notes URLs: Additional Info: CRITICAL - Socket timeout after 10 seconds ___ Betacluster-alerts mailing list

[Betacluster-alerts] Host DOWN alert for deployment-kafka-jumbo-1!

2018-11-19 Thread shinken
Notification Type: PROBLEM Host: deployment-kafka-jumbo-1 State: DOWN Address: 10.68.23.243 Info: CRITICAL - Host Unreachable (10.68.23.243) Date/Time: Tue 20 Nov 00:17:40 UTC 2018 ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-sca02/SSH is CRITICAL **

2018-11-19 Thread shinken
Notification Type: PROBLEM Service: SSH Host: deployment-sca02 Address: 10.68.20.153 State: CRITICAL Date/Time: Tue 20 Nov 00:17:49 UTC 2018 Notes URLs: Additional Info: CRITICAL - Socket timeout after 10 seconds ___ Betacluster-alerts mailing

[Betacluster-alerts] Host DOWN alert for deployment-deploy02!

2018-11-19 Thread shinken
Notification Type: PROBLEM Host: deployment-deploy02 State: DOWN Address: 10.68.23.98 Info: CRITICAL - Host Unreachable (10.68.23.98) Date/Time: Tue 20 Nov 00:17:28 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] Host DOWN alert for deployment-puppetdb02!

2018-11-19 Thread shinken
Notification Type: PROBLEM Host: deployment-puppetdb02 State: DOWN Address: 10.68.19.126 Info: CRITICAL - Host Unreachable (10.68.19.126) Date/Time: Tue 20 Nov 00:17:24 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** PROBLEM alert - deployment-zookeeper02/SSH is CRITICAL **

2018-11-19 Thread shinken
Notification Type: PROBLEM Service: SSH Host: deployment-zookeeper02 Address: 10.68.18.75 State: CRITICAL Date/Time: Tue 20 Nov 00:16:37 UTC 2018 Notes URLs: Additional Info: CRITICAL - Socket timeout after 10 seconds ___ Betacluster-alerts

[Betacluster-alerts] ** PROBLEM alert - deployment-sca04/SSH is CRITICAL **

2018-11-19 Thread shinken
Notification Type: PROBLEM Service: SSH Host: deployment-sca04 Address: 10.68.18.80 State: CRITICAL Date/Time: Tue 20 Nov 00:16:50 UTC 2018 Notes URLs: Additional Info: CRITICAL - Socket timeout after 10 seconds ___ Betacluster-alerts mailing list

[Betacluster-alerts] Host DOWN alert for deployment-webperf12!

2018-11-19 Thread shinken
Notification Type: PROBLEM Host: deployment-webperf12 State: DOWN Address: 10.68.20.36 Info: CRITICAL - Host Unreachable (10.68.20.36) Date/Time: Tue 20 Nov 00:16:05 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** PROBLEM alert - deployment-fluorine02/SSH is CRITICAL **

2018-11-19 Thread shinken
Notification Type: PROBLEM Service: SSH Host: deployment-fluorine02 Address: 10.68.23.106 State: CRITICAL Date/Time: Tue 20 Nov 00:16:32 UTC 2018 Notes URLs: Additional Info: CRITICAL - Socket timeout after 10 seconds ___ Betacluster-alerts

[Betacluster-alerts] ** PROBLEM alert - Generic Beta Cluster/English Wikipedia Main page is CRITICAL **

2018-11-19 Thread shinken
Notification Type: PROBLEM Service: English Wikipedia Main page Host: Generic Beta Cluster Address: en.wikipedia.beta.wmflabs.org State: CRITICAL Date/Time: Tue 20 Nov 01:15:38 UTC 2018 Notes URLs: Additional Info: CRITICAL - Socket timeout after 10 seconds

[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is CRITICAL **

2018-11-19 Thread shinken
Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: CRITICAL Date/Time: Tue 20 Nov 01:46:21 UTC 2018 Notes URLs: Additional Info: CRITICAL: 20.00% of data above the critical threshold [10.0]

[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is WARNING **

2018-11-19 Thread shinken
Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: WARNING Date/Time: Tue 20 Nov 00:25:22 UTC 2018 Notes URLs: Additional Info: WARNING: 20.00% of data above the warning threshold [1.0]

[Betacluster-alerts] selenium-daily-beta-MediaWiki - Build # 88 - Still Failing!

2018-11-19 Thread jenkins-bot
FAILURE: selenium-daily-beta-MediaWiki Build #88 (Tue, 20 Nov 2018 07:40:00 +) Test Result 3 failed, 0 skipped Failed Tests Test NameDurationAge chrome.BlankPage2."before each" hook540 sec35

[Betacluster-alerts] ** PROBLEM alert - deployment-deploy01/Free space - all mounts is WARNING **

2018-11-19 Thread shinken
Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-deploy01 Address: 10.68.23.38 State: WARNING Date/Time: Mon 19 Nov 17:23:16 UTC 2018 Notes URLs: Additional Info: WARNING: deployment-prep.deployment-deploy01.diskspace.root.byte_percentfree (<11.11%)

[Betacluster-alerts] ** RECOVERY alert - deployment-deploy01/Free space - all mounts is OK **

2018-11-19 Thread shinken
Notification Type: RECOVERY Service: Free space - all mounts Host: deployment-deploy01 Address: 10.68.23.38 State: OK Date/Time: Mon 19 Nov 17:33:18 UTC 2018 Notes URLs: Additional Info: OK: All targets OK ___ Betacluster-alerts mailing list

[Betacluster-alerts] Host DOWN alert for deployment-maps04!

2018-11-19 Thread shinken
Notification Type: PROBLEM Host: deployment-maps04 State: DOWN Address: 10.68.19.18 Info: CRITICAL - Host Unreachable (10.68.19.18) Date/Time: Mon 19 Nov 18:23:53 UTC 2018 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] beta-code-update-eqiad - Build # 225524 - Failure!

2018-11-19 Thread jenkins-bot
beta-code-update-eqiad - Build # 225524 - Failure: Check console output at https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/225524/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] Host DOWN alert for deployment-urldownloader02!

2018-11-19 Thread shinken
Notification Type: PROBLEM Host: deployment-urldownloader02 State: DOWN Address: 10.68.19.117 Info: CRITICAL - Host Unreachable (10.68.19.117) Date/Time: Mon 19 Nov 18:24:56 UTC 2018 ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-deploy01/Free space - all mounts is WARNING **

2018-11-19 Thread shinken
Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-deploy01 Address: 10.68.23.38 State: WARNING Date/Time: Mon 19 Nov 08:19:16 UTC 2018 Notes URLs: Additional Info: WARNING: deployment-prep.deployment-deploy01.diskspace.root.byte_percentfree (<11.11%)

[Betacluster-alerts] ** RECOVERY alert - deployment-deploy01/Free space - all mounts is OK **

2018-11-19 Thread shinken
Notification Type: RECOVERY Service: Free space - all mounts Host: deployment-deploy01 Address: 10.68.23.38 State: OK Date/Time: Mon 19 Nov 08:29:18 UTC 2018 Notes URLs: Additional Info: OK: All targets OK ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-deploy01/Free space - all mounts is WARNING **

2018-11-19 Thread shinken
Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-deploy01 Address: 10.68.23.38 State: WARNING Date/Time: Mon 19 Nov 09:55:17 UTC 2018 Notes URLs: Additional Info: WARNING: deployment-prep.deployment-deploy01.diskspace.root.byte_percentfree (<11.11%)

[Betacluster-alerts] ** RECOVERY alert - deployment-deploy01/Free space - all mounts is OK **

2018-11-19 Thread shinken
Notification Type: RECOVERY Service: Free space - all mounts Host: deployment-deploy01 Address: 10.68.23.38 State: OK Date/Time: Mon 19 Nov 10:05:18 UTC 2018 Notes URLs: Additional Info: OK: All targets OK ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-deploy01/Free space - all mounts is WARNING **

2018-11-19 Thread shinken
Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-deploy01 Address: 10.68.23.38 State: WARNING Date/Time: Mon 19 Nov 10:37:16 UTC 2018 Notes URLs: Additional Info: WARNING: deployment-prep.deployment-deploy01.diskspace.root.byte_percentfree (<11.11%)

[Betacluster-alerts] ** PROBLEM alert - deployment-deploy01/Free space - all mounts is WARNING **

2018-11-19 Thread shinken
Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-deploy01 Address: 10.68.23.38 State: WARNING Date/Time: Mon 19 Nov 12:28:19 UTC 2018 Notes URLs: Additional Info: WARNING: deployment-prep.deployment-deploy01.diskspace.root.byte_percentfree (<22.22%)

[Betacluster-alerts] ** RECOVERY alert - deployment-deploy01/Free space - all mounts is OK **

2018-11-19 Thread shinken
Notification Type: RECOVERY Service: Free space - all mounts Host: deployment-deploy01 Address: 10.68.23.38 State: OK Date/Time: Mon 19 Nov 12:38:17 UTC 2018 Notes URLs: Additional Info: OK: All targets OK ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-deploy01/Free space - all mounts is WARNING **

2018-11-19 Thread shinken
Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-deploy01 Address: 10.68.23.38 State: WARNING Date/Time: Mon 19 Nov 13:14:18 UTC 2018 Notes URLs: Additional Info: WARNING: deployment-prep.deployment-deploy01.diskspace.root.byte_percentfree (<11.11%)

[Betacluster-alerts] beta-scap-eqiad - Build # 228211 - Failure!

2018-11-19 Thread jenkins-bot
beta-scap-eqiad - Build # 228211 - Failure: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/228211/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** RECOVERY alert - deployment-deploy01/Free space - all mounts is OK **

2018-11-19 Thread shinken
Notification Type: RECOVERY Service: Free space - all mounts Host: deployment-deploy01 Address: 10.68.23.38 State: OK Date/Time: Mon 19 Nov 13:24:16 UTC 2018 Notes URLs: Additional Info: OK: All targets OK ___ Betacluster-alerts mailing list

[Betacluster-alerts] beta-scap-eqiad - Build # 228212 - Fixed!

2018-11-19 Thread jenkins-bot
beta-scap-eqiad - Build # 228212 - Fixed: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/228212/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** RECOVERY alert - deployment-logstash2/SSH is OK **

2018-11-19 Thread shinken
Notification Type: RECOVERY Service: SSH Host: deployment-logstash2 Address: 10.68.16.147 State: OK Date/Time: Mon 19 Nov 15:12:53 UTC 2018 Notes URLs: Additional Info: SSH OK - OpenSSH_6.7p1 Debian-5+deb8u7 (protocol 2.0) ___ Betacluster-alerts