[Betacluster-alerts] ** PROBLEM alert - deployment-mediawiki06/Puppet errors is CRITICAL **

2017-07-06 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-mediawiki06 Address: 10.68.19.241 State: CRITICAL Date/Time: Thu 06 Jul 16:58:58 UTC 2017 Additional Info: CRITICAL: 30.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** RECOVERY alert - deployment-mediawiki06/Puppet errors is OK **

2017-07-06 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-mediawiki06 Address: 10.68.19.241 State: OK Date/Time: Thu 06 Jul 18:09:00 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-mediawiki04/Puppet errors is CRITICAL **

2017-07-06 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-mediawiki04 Address: 10.68.19.128 State: CRITICAL Date/Time: Thu 06 Jul 17:16:24 UTC 2017 Additional Info: CRITICAL: 55.56% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - Generic Beta Cluster/English Wikipedia Mobile Main page is CRITICAL **

2017-07-06 Thread shinken
Notification Type: PROBLEM Service: English Wikipedia Mobile Main page Host: Generic Beta Cluster Address: en.wikipedia.beta.wmflabs.org State: CRITICAL Date/Time: Thu 06 Jul 20:24:57 UTC 2017 Additional Info: CRITICAL - Socket timeout after 10 seconds

[Betacluster-alerts] ** PROBLEM alert - deployment-jobrunner02/Puppet errors is CRITICAL **

2017-07-06 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-jobrunner02 Address: 10.68.19.42 State: CRITICAL Date/Time: Thu 06 Jul 17:24:44 UTC 2017 Additional Info: CRITICAL: 40.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** RECOVERY alert - deployment-mediawiki04/Puppet errors is OK **

2017-07-06 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-mediawiki04 Address: 10.68.19.128 State: OK Date/Time: Thu 06 Jul 17:51:24 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] beta-scap-eqiad - Build # 162868 - Fixed!

2017-07-06 Thread jenkins-bot
beta-scap-eqiad - Build # 162868 - Fixed: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/162868/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** RECOVERY alert - deployment-mediawiki04/App Server Main HTTP Response is OK **

2017-07-06 Thread shinken
Notification Type: RECOVERY Service: App Server Main HTTP Response Host: deployment-mediawiki04 Address: 10.68.19.128 State: OK Date/Time: Thu 06 Jul 20:16:50 UTC 2017 Additional Info: HTTP OK: HTTP/1.1 200 OK - 54172 bytes in 1.957 second response time

[Betacluster-alerts] ** PROBLEM alert - deployment-tmh01/Puppet errors is CRITICAL **

2017-07-06 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-tmh01 Address: 10.68.16.211 State: CRITICAL Date/Time: Thu 06 Jul 17:03:33 UTC 2017 Additional Info: CRITICAL: 44.44% of data above the critical threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** PROBLEM alert - deployment-mediawiki05/Puppet errors is CRITICAL **

2017-07-06 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-mediawiki05 Address: 10.68.22.21 State: CRITICAL Date/Time: Thu 06 Jul 17:21:04 UTC 2017 Additional Info: CRITICAL: 55.56% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** RECOVERY alert - deployment-jobrunner02/Puppet errors is OK **

2017-07-06 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-jobrunner02 Address: 10.68.19.42 State: OK Date/Time: Thu 06 Jul 18:04:45 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] beta-scap-eqiad - Build # 162867 - Failure!

2017-07-06 Thread jenkins-bot
beta-scap-eqiad - Build # 162867 - Failure: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/162867/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** PROBLEM alert - Generic Beta Cluster/English Wikipedia Main page is CRITICAL **

2017-07-06 Thread shinken
Notification Type: PROBLEM Service: English Wikipedia Main page Host: Generic Beta Cluster Address: en.wikipedia.beta.wmflabs.org State: CRITICAL Date/Time: Thu 06 Jul 20:05:31 UTC 2017 Additional Info: CRITICAL - Socket timeout after 10 seconds ___

[Betacluster-alerts] ** RECOVERY alert - deployment-mediawiki04/Puppet errors is OK **

2017-07-06 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-mediawiki04 Address: 10.68.19.128 State: OK Date/Time: Thu 06 Jul 16:50:23 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - Generic Beta Cluster/English Wikipedia Main page is OK **

2017-07-06 Thread shinken
Notification Type: RECOVERY Service: English Wikipedia Main page Host: Generic Beta Cluster Address: en.wikipedia.beta.wmflabs.org State: OK Date/Time: Thu 06 Jul 20:10:23 UTC 2017 Additional Info: HTTP OK: HTTP/1.1 200 OK - 54641 bytes in 2.362 second response time

[Betacluster-alerts] ** RECOVERY alert - Generic Beta Cluster/English Wikipedia Mobile Main page is OK **

2017-07-06 Thread shinken
Notification Type: RECOVERY Service: English Wikipedia Mobile Main page Host: Generic Beta Cluster Address: en.wikipedia.beta.wmflabs.org State: OK Date/Time: Thu 06 Jul 20:18:51 UTC 2017 Additional Info: HTTP OK: HTTP/1.1 200 OK - 43070 bytes in 2.073 second response time

[Betacluster-alerts] ** RECOVERY alert - deployment-mira/Puppet errors is OK **

2017-07-06 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-mira Address: 10.68.20.135 State: OK Date/Time: Thu 06 Jul 17:32:41 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-mediawiki05/Puppet errors is OK **

2017-07-06 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-mediawiki05 Address: 10.68.22.21 State: OK Date/Time: Thu 06 Jul 17:56:04 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - Graphite Labs/Mediawiki Error Rate is OK **

2017-07-06 Thread shinken
Notification Type: RECOVERY Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: OK Date/Time: Thu 06 Jul 19:20:09 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [1.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - Generic Beta Cluster/English Wikipedia Mobile Main page is OK **

2017-07-06 Thread shinken
Notification Type: RECOVERY Service: English Wikipedia Mobile Main page Host: Generic Beta Cluster Address: en.wikipedia.beta.wmflabs.org State: OK Date/Time: Thu 06 Jul 20:29:48 UTC 2017 Additional Info: HTTP OK: HTTP/1.1 200 OK - 43080 bytes in 2.087 second response time

[Betacluster-alerts] ** PROBLEM alert - deployment-pdfrender02/Puppet errors is CRITICAL **

2017-07-06 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-pdfrender02 Address: 10.68.21.240 State: CRITICAL Date/Time: Thu 06 Jul 21:15:31 UTC 2017 Additional Info: CRITICAL: 22.22% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** RECOVERY alert - deployment-pdfrender02/Puppet errors is OK **

2017-07-06 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-pdfrender02 Address: 10.68.21.240 State: OK Date/Time: Thu 06 Jul 21:55:31 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-fluorine02/Free space - all mounts is OK **

2017-07-06 Thread shinken
Notification Type: RECOVERY Service: Free space - all mounts Host: deployment-fluorine02 Address: 10.68.23.106 State: OK Date/Time: Thu 06 Jul 07:15:08 UTC 2017 Additional Info: OK: All targets OK ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - Graphite Labs/Mediawiki Error Rate is OK **

2017-07-06 Thread shinken
Notification Type: RECOVERY Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: OK Date/Time: Thu 06 Jul 07:30:10 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [1.0] ___ Betacluster-alerts

[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is WARNING **

2017-07-06 Thread shinken
Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: WARNING Date/Time: Thu 06 Jul 12:41:12 UTC 2017 Additional Info: WARNING: 80.00% of data above the warning threshold [1.0] ___

[Betacluster-alerts] ** RECOVERY alert - Graphite Labs/Mediawiki Error Rate is OK **

2017-07-06 Thread shinken
Notification Type: RECOVERY Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: OK Date/Time: Thu 06 Jul 12:46:10 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [1.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - Graphite Labs/Mediawiki Error Rate is OK **

2017-07-06 Thread shinken
Notification Type: RECOVERY Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: OK Date/Time: Thu 06 Jul 13:47:11 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [1.0] ___ Betacluster-alerts

[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is WARNING **

2017-07-06 Thread shinken
Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: WARNING Date/Time: Thu 06 Jul 14:08:10 UTC 2017 Additional Info: WARNING: 80.00% of data above the warning threshold [1.0] ___

[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is WARNING **

2017-07-06 Thread shinken
Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: WARNING Date/Time: Thu 06 Jul 13:32:10 UTC 2017 Additional Info: WARNING: 80.00% of data above the warning threshold [1.0] ___

[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is WARNING **

2017-07-06 Thread shinken
Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: WARNING Date/Time: Thu 06 Jul 14:29:12 UTC 2017 Additional Info: WARNING: 40.00% of data above the warning threshold [1.0] ___

[Betacluster-alerts] ** RECOVERY alert - deployment-cache-upload04/Puppet errors is OK **

2017-07-06 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-cache-upload04 Address: 10.68.18.109 State: OK Date/Time: Thu 06 Jul 15:40:52 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing

[Betacluster-alerts] ** PROBLEM alert - deployment-cache-upload04/Puppet errors is CRITICAL **

2017-07-06 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-cache-upload04 Address: 10.68.18.109 State: CRITICAL Date/Time: Thu 06 Jul 15:09:12 UTC 2017 Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-ms-be03/Puppet errors is CRITICAL **

2017-07-06 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-ms-be03 Address: 10.68.22.125 State: CRITICAL Date/Time: Thu 06 Jul 16:13:10 UTC 2017 Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-ms-be04/Puppet errors is CRITICAL **

2017-07-06 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-ms-be04 Address: 10.68.16.139 State: CRITICAL Date/Time: Thu 06 Jul 16:20:32 UTC 2017 Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-mediawiki04/Puppet errors is CRITICAL **

2017-07-06 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-mediawiki04 Address: 10.68.19.128 State: CRITICAL Date/Time: Thu 06 Jul 16:35:24 UTC 2017 Additional Info: CRITICAL: 44.44% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-mediawiki04/App Server Main HTTP Response is CRITICAL **

2017-07-06 Thread shinken
Notification Type: PROBLEM Service: App Server Main HTTP Response Host: deployment-mediawiki04 Address: 10.68.19.128 State: CRITICAL Date/Time: Thu 06 Jul 20:11:56 UTC 2017 Additional Info: CRITICAL - Socket timeout after 10 seconds ___

[Betacluster-alerts] ** PROBLEM alert - deployment-fluorine02/Free space - all mounts is CRITICAL **

2017-07-06 Thread shinken
Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-fluorine02 Address: 10.68.23.106 State: CRITICAL Date/Time: Fri 07 Jul 01:56:08 UTC 2017 Additional Info: CRITICAL: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree (<22.22%)

[Betacluster-alerts] ** RECOVERY alert - deployment-pdfrender02/Puppet errors is OK **

2017-07-06 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-pdfrender02 Address: 10.68.21.240 State: OK Date/Time: Fri 07 Jul 01:56:32 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-pdfrender02/Puppet errors is CRITICAL **

2017-07-06 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-pdfrender02 Address: 10.68.21.240 State: CRITICAL Date/Time: Fri 07 Jul 01:16:30 UTC 2017 Additional Info: CRITICAL: 40.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-mira/Puppet errors is CRITICAL **

2017-07-06 Thread shinken
Notification Type: PROBLEM Service: Puppet errors Host: deployment-mira Address: 10.68.20.135 State: CRITICAL Date/Time: Thu 06 Jul 16:57:41 UTC 2017 Additional Info: CRITICAL: 66.67% of data above the critical threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - deployment-tmh01/Puppet errors is OK **

2017-07-06 Thread shinken
Notification Type: RECOVERY Service: Puppet errors Host: deployment-tmh01 Address: 10.68.16.211 State: OK Date/Time: Thu 06 Jul 17:38:32 UTC 2017 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - Generic Beta Cluster/English Wikipedia Mobile Main page is CRITICAL **

2017-07-06 Thread shinken
Notification Type: PROBLEM Service: English Wikipedia Mobile Main page Host: Generic Beta Cluster Address: en.wikipedia.beta.wmflabs.org State: CRITICAL Date/Time: Thu 06 Jul 20:13:57 UTC 2017 Additional Info: CRITICAL - Socket timeout after 10 seconds

[Betacluster-alerts] ** PROBLEM alert - deployment-fluorine02/Free space - all mounts is WARNING **

2017-07-06 Thread shinken
Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-fluorine02 Address: 10.68.23.106 State: WARNING Date/Time: Thu 06 Jul 23:01:08 UTC 2017 Additional Info: WARNING: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree (<66.67%)