[Betacluster-alerts] ** RECOVERY alert - deployment-eventlog02/Free space - all mounts is OK **
Notification Type: RECOVERY Service: Free space - all mounts Host: deployment-eventlog02 Address: 10.68.18.138 State: OK Date/Time: Tue 19 Dec 06:55:11 UTC 2017 Notes URLs: Additional Info: OK: All targets OK ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** RECOVERY alert - deployment-fluorine02/Free space - all mounts is OK **
Notification Type: RECOVERY Service: Free space - all mounts Host: deployment-fluorine02 Address: 10.68.23.106 State: OK Date/Time: Tue 19 Dec 06:54:00 UTC 2017 Notes URLs: Additional Info: OK: All targets OK ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** RECOVERY alert - deployment-kafka03/Free space - all mounts is OK **
Notification Type: RECOVERY Service: Free space - all mounts Host: deployment-kafka03 Address: 10.68.16.138 State: OK Date/Time: Tue 19 Dec 06:52:27 UTC 2017 Notes URLs: Additional Info: OK: All targets OK ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-secureredirexperiment/Puppet staleness is CRITICAL **
Notification Type: PROBLEM Service: Puppet staleness Host: deployment-secureredirexperiment Address: 10.68.17.132 State: CRITICAL Date/Time: Tue 19 Dec 06:51:48 UTC 2017 Notes URLs: Additional Info: CRITICAL: 10.00% of data above the critical threshold [43200.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-ms-be04/Puppet staleness is CRITICAL **
Notification Type: PROBLEM Service: Puppet staleness Host: deployment-ms-be04 Address: 10.68.16.139 State: CRITICAL Date/Time: Tue 19 Dec 06:49:24 UTC 2017 Notes URLs: Additional Info: CRITICAL: 20.00% of data above the critical threshold [43200.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-ms-be03/Puppet staleness is CRITICAL **
Notification Type: PROBLEM Service: Puppet staleness Host: deployment-ms-be03 Address: 10.68.22.125 State: CRITICAL Date/Time: Tue 19 Dec 06:44:37 UTC 2017 Notes URLs: Additional Info: CRITICAL: 33.33% of data above the critical threshold [43200.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-restbase02/Puppet staleness is CRITICAL **
Notification Type: PROBLEM Service: Puppet staleness Host: deployment-restbase02 Address: 10.68.17.189 State: CRITICAL Date/Time: Tue 19 Dec 05:22:25 UTC 2017 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [43200.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-restbase01/Puppet staleness is CRITICAL **
Notification Type: PROBLEM Service: Puppet staleness Host: deployment-restbase01 Address: 10.68.16.128 State: CRITICAL Date/Time: Tue 19 Dec 04:59:27 UTC 2017 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [43200.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is WARNING **
Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: WARNING Date/Time: Tue 19 Dec 04:32:37 UTC 2017 Notes URLs: Additional Info: WARNING: 60.00% of data above the warning threshold [1.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** RECOVERY alert - Graphite Labs/Mediawiki Error Rate is OK **
Notification Type: RECOVERY Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: OK Date/Time: Tue 19 Dec 04:26:37 UTC 2017 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [1.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-kafka03/Free space - all mounts is WARNING **
Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-kafka03 Address: 10.68.16.138 State: WARNING Date/Time: Tue 19 Dec 04:02:27 UTC 2017 Notes URLs: Additional Info: WARNING: deployment-prep.deployment-kafka03.diskspace.root.byte_percentfree (<66.67%) ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** RECOVERY alert - deployment-kafka03/Free space - all mounts is OK **
Notification Type: RECOVERY Service: Free space - all mounts Host: deployment-kafka03 Address: 10.68.16.138 State: OK Date/Time: Tue 19 Dec 03:31:26 UTC 2017 Notes URLs: Additional Info: OK: All targets OK ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-kafka03/Free space - all mounts is WARNING **
Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-kafka03 Address: 10.68.16.138 State: WARNING Date/Time: Tue 19 Dec 03:26:27 UTC 2017 Notes URLs: Additional Info: WARNING: deployment-prep.deployment-kafka03.diskspace.root.byte_percentfree (<11.11%) ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-fluorine02/Free space - all mounts is CRITICAL **
Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-fluorine02 Address: 10.68.23.106 State: CRITICAL Date/Time: Tue 19 Dec 03:03:59 UTC 2017 Notes URLs: Additional Info: CRITICAL: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree (<30.00%) ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-tin/Puppet errors is CRITICAL **
Notification Type: PROBLEM Service: Puppet errors Host: deployment-tin Address: 10.68.21.205 State: CRITICAL Date/Time: Tue 19 Dec 02:49:20 UTC 2017 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-mira/Puppet errors is CRITICAL **
Notification Type: PROBLEM Service: Puppet errors Host: deployment-mira Address: 10.68.20.135 State: CRITICAL Date/Time: Tue 19 Dec 02:43:29 UTC 2017 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-puppetmaster02/Long lived cherry-picks on puppetmaster is CRITICAL **
Notification Type: PROBLEM Service: Long lived cherry-picks on puppetmaster Host: deployment-puppetmaster02 Address: 10.68.21.200 State: CRITICAL Date/Time: Tue 19 Dec 02:30:44 UTC 2017 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-mx/Free space - all mounts is WARNING **
Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-mx Address: 10.68.17.78 State: WARNING Date/Time: Tue 19 Dec 02:04:07 UTC 2017 Notes URLs: Additional Info: WARNING: deployment-prep.deployment-mx.diskspace._var_log.byte_percentfree (<100.00%) ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-eventlog02/Free space - all mounts is WARNING **
Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-eventlog02 Address: 10.68.18.138 State: WARNING Date/Time: Tue 19 Dec 01:40:11 UTC 2017 Notes URLs: Additional Info: WARNING: deployment-prep.deployment-eventlog02.diskspace.root.byte_percentfree (<33.33%) ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is WARNING **
Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: WARNING Date/Time: Tue 19 Dec 00:21:37 UTC 2017 Notes URLs: Additional Info: WARNING: 20.00% of data above the warning threshold [1.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is CRITICAL **
Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: CRITICAL Date/Time: Tue 19 Dec 00:16:36 UTC 2017 Notes URLs: Additional Info: CRITICAL: 20.00% of data above the critical threshold [10.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-fluorine02/Free space - all mounts is WARNING **
Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-fluorine02 Address: 10.68.23.106 State: WARNING Date/Time: Tue 19 Dec 00:09:00 UTC 2017 Notes URLs: Additional Info: WARNING: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree (<40.00%) ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-kafka-jumbo-1/Puppet errors is CRITICAL **
Notification Type: PROBLEM Service: Puppet errors Host: deployment-kafka-jumbo-1 Address: 10.68.23.243 State: CRITICAL Date/Time: Mon 18 Dec 23:37:24 UTC 2017 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-netbox/Puppet errors is CRITICAL **
Notification Type: PROBLEM Service: Puppet errors Host: deployment-netbox Address: 10.68.19.203 State: CRITICAL Date/Time: Mon 18 Dec 23:36:09 UTC 2017 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-kafka-jumbo-2/Puppet errors is CRITICAL **
Notification Type: PROBLEM Service: Puppet errors Host: deployment-kafka-jumbo-2 Address: 10.68.16.87 State: CRITICAL Date/Time: Mon 18 Dec 23:30:56 UTC 2017 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-etcd-01/Puppet errors is CRITICAL **
Notification Type: PROBLEM Service: Puppet errors Host: deployment-etcd-01 Address: 10.68.19.227 State: CRITICAL Date/Time: Mon 18 Dec 22:35:42 UTC 2017 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-sentry01/Puppet errors is CRITICAL **
Notification Type: PROBLEM Service: Puppet errors Host: deployment-sentry01 Address: 10.68.19.148 State: CRITICAL Date/Time: Mon 18 Dec 22:20:13 UTC 2017 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-mediawiki05/App Server Main HTTP Response is CRITICAL **
Notification Type: PROBLEM Service: App Server Main HTTP Response Host: deployment-mediawiki05 Address: 10.68.22.21 State: CRITICAL Date/Time: Mon 18 Dec 21:31:31 UTC 2017 Notes URLs: Additional Info: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 23217 bytes in 0.579 second response time ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is WARNING **
Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: WARNING Date/Time: Mon 18 Dec 21:31:36 UTC 2017 Notes URLs: Additional Info: WARNING: 60.00% of data above the warning threshold [1.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - Generic Beta Cluster/English Wikipedia Mobile Main page is CRITICAL **
Notification Type: PROBLEM Service: English Wikipedia Mobile Main page Host: Generic Beta Cluster Address: en.wikipedia.beta.wmflabs.org State: CRITICAL Date/Time: Mon 18 Dec 21:31:15 UTC 2017 Notes URLs: Additional Info: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 19229 bytes in 0.873 second response time ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-mediawiki04/App Server Main HTTP Response is CRITICAL **
Notification Type: PROBLEM Service: App Server Main HTTP Response Host: deployment-mediawiki04 Address: 10.68.19.128 State: CRITICAL Date/Time: Mon 18 Dec 21:30:34 UTC 2017 Notes URLs: Additional Info: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 23217 bytes in 0.745 second response time ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-mediawiki06/App Server Main HTTP Response is CRITICAL **
Notification Type: PROBLEM Service: App Server Main HTTP Response Host: deployment-mediawiki06 Address: 10.68.19.241 State: CRITICAL Date/Time: Mon 18 Dec 21:28:49 UTC 2017 Notes URLs: Additional Info: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 23217 bytes in 0.670 second response time ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - Generic Beta Cluster/English Wikipedia Main page is CRITICAL **
Notification Type: PROBLEM Service: English Wikipedia Main page Host: Generic Beta Cluster Address: en.wikipedia.beta.wmflabs.org State: CRITICAL Date/Time: Mon 18 Dec 21:28:43 UTC 2017 Notes URLs: Additional Info: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 23770 bytes in 0.749 second response time ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** RECOVERY alert - Graphite Labs/Mediawiki Error Rate is OK **
Notification Type: RECOVERY Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: OK Date/Time: Mon 18 Dec 20:54:37 UTC 2017 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [1.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is WARNING **
Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: WARNING Date/Time: Mon 18 Dec 20:39:36 UTC 2017 Notes URLs: Additional Info: WARNING: 80.00% of data above the warning threshold [1.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-eventlogging04/Puppet errors is CRITICAL **
Notification Type: PROBLEM Service: Puppet errors Host: deployment-eventlogging04 Address: 10.68.23.204 State: CRITICAL Date/Time: Mon 18 Dec 20:34:28 UTC 2017 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** RECOVERY alert - deployment-sentry01/Puppet staleness is OK **
Notification Type: RECOVERY Service: Puppet staleness Host: deployment-sentry01 Address: 10.68.19.148 State: OK Date/Time: Mon 18 Dec 19:56:29 UTC 2017 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [3600.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-secureredirexperiment/Puppet staleness is WARNING **
Notification Type: PROBLEM Service: Puppet staleness Host: deployment-secureredirexperiment Address: 10.68.17.132 State: WARNING Date/Time: Mon 18 Dec 19:56:48 UTC 2017 Notes URLs: Additional Info: WARNING: 60.00% of data above the warning threshold [3600.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** RECOVERY alert - deployment-tmh01/Puppet staleness is OK **
Notification Type: RECOVERY Service: Puppet staleness Host: deployment-tmh01 Address: 10.68.16.211 State: OK Date/Time: Mon 18 Dec 19:54:11 UTC 2017 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [3600.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** RECOVERY alert - deployment-db04/Puppet staleness is OK **
Notification Type: RECOVERY Service: Puppet staleness Host: deployment-db04 Address: 10.68.18.35 State: OK Date/Time: Mon 18 Dec 19:50:41 UTC 2017 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [3600.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is CRITICAL **
Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: CRITICAL Date/Time: Mon 18 Dec 19:49:36 UTC 2017 Notes URLs: Additional Info: CRITICAL: 20.00% of data above the critical threshold [10.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-ms-be04/Puppet staleness is WARNING **
Notification Type: PROBLEM Service: Puppet staleness Host: deployment-ms-be04 Address: 10.68.16.139 State: WARNING Date/Time: Mon 18 Dec 19:49:24 UTC 2017 Notes URLs: Additional Info: WARNING: 20.00% of data above the warning threshold [3600.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** RECOVERY alert - deployment-memc06/Puppet staleness is OK **
Notification Type: RECOVERY Service: Puppet staleness Host: deployment-memc06 Address: 10.68.22.239 State: OK Date/Time: Mon 18 Dec 19:48:45 UTC 2017 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [3600.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** RECOVERY alert - deployment-cpjobqueue/Puppet staleness is OK **
Notification Type: RECOVERY Service: Puppet staleness Host: deployment-cpjobqueue Address: 10.68.22.161 State: OK Date/Time: Mon 18 Dec 19:46:58 UTC 2017 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [3600.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** RECOVERY alert - deployment-elastic05/Puppet staleness is OK **
Notification Type: RECOVERY Service: Puppet staleness Host: deployment-elastic05 Address: 10.68.20.21 State: OK Date/Time: Mon 18 Dec 19:46:49 UTC 2017 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [3600.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** RECOVERY alert - deployment-etcd-01/Puppet staleness is OK **
Notification Type: RECOVERY Service: Puppet staleness Host: deployment-etcd-01 Address: 10.68.19.227 State: OK Date/Time: Mon 18 Dec 19:46:22 UTC 2017 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [3600.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** RECOVERY alert - deployment-aqs01/Puppet staleness is OK **
Notification Type: RECOVERY Service: Puppet staleness Host: deployment-aqs01 Address: 10.68.18.237 State: OK Date/Time: Mon 18 Dec 19:46:13 UTC 2017 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [3600.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-ms-be03/Puppet staleness is WARNING **
Notification Type: PROBLEM Service: Puppet staleness Host: deployment-ms-be03 Address: 10.68.22.125 State: WARNING Date/Time: Mon 18 Dec 19:44:37 UTC 2017 Notes URLs: Additional Info: WARNING: 33.33% of data above the warning threshold [3600.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-tmh01/Puppet staleness is WARNING **
Notification Type: PROBLEM Service: Puppet staleness Host: deployment-tmh01 Address: 10.68.16.211 State: WARNING Date/Time: Mon 18 Dec 19:44:09 UTC 2017 Notes URLs: Additional Info: WARNING: 12.50% of data above the warning threshold [3600.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] beta-code-update-eqiad - Build # 185792 - Fixed!
beta-code-update-eqiad - Build # 185792 - Fixed: Check console output at https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/185792/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** RECOVERY alert - deployment-parsoid09/Puppet staleness is OK **
Notification Type: RECOVERY Service: Puppet staleness Host: deployment-parsoid09 Address: 10.68.20.142 State: OK Date/Time: Mon 18 Dec 19:43:44 UTC 2017 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [3600.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-elastic05/Puppet staleness is WARNING **
Notification Type: PROBLEM Service: Puppet staleness Host: deployment-elastic05 Address: 10.68.20.21 State: WARNING Date/Time: Mon 18 Dec 19:41:49 UTC 2017 Notes URLs: Additional Info: WARNING: 11.11% of data above the warning threshold [3600.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-aqs01/Puppet staleness is WARNING **
Notification Type: PROBLEM Service: Puppet staleness Host: deployment-aqs01 Address: 10.68.18.237 State: WARNING Date/Time: Mon 18 Dec 19:41:12 UTC 2017 Notes URLs: Additional Info: WARNING: 11.11% of data above the warning threshold [3600.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-db04/Puppet staleness is WARNING **
Notification Type: PROBLEM Service: Puppet staleness Host: deployment-db04 Address: 10.68.18.35 State: WARNING Date/Time: Mon 18 Dec 19:40:42 UTC 2017 Notes URLs: Additional Info: WARNING: 10.00% of data above the warning threshold [3600.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** RECOVERY alert - deployment-cache-upload04/Puppet staleness is OK **
Notification Type: RECOVERY Service: Puppet staleness Host: deployment-cache-upload04 Address: 10.68.18.109 State: OK Date/Time: Mon 18 Dec 19:39:50 UTC 2017 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [3600.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** RECOVERY alert - deployment-trending01/Puppet staleness is OK **
Notification Type: RECOVERY Service: Puppet staleness Host: deployment-trending01 Address: 10.68.18.186 State: OK Date/Time: Mon 18 Dec 19:39:48 UTC 2017 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [3600.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** RECOVERY alert - deployment-sca03/Puppet staleness is OK **
Notification Type: RECOVERY Service: Puppet staleness Host: deployment-sca03 Address: 10.68.21.183 State: OK Date/Time: Mon 18 Dec 19:39:45 UTC 2017 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [3600.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-memc06/Puppet staleness is WARNING **
Notification Type: PROBLEM Service: Puppet staleness Host: deployment-memc06 Address: 10.68.22.239 State: WARNING Date/Time: Mon 18 Dec 19:38:47 UTC 2017 Notes URLs: Additional Info: WARNING: 10.00% of data above the warning threshold [3600.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-cpjobqueue/Puppet staleness is WARNING **
Notification Type: PROBLEM Service: Puppet staleness Host: deployment-cpjobqueue Address: 10.68.22.161 State: WARNING Date/Time: Mon 18 Dec 19:36:57 UTC 2017 Notes URLs: Additional Info: WARNING: 10.00% of data above the warning threshold [3600.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-etcd-01/Puppet staleness is WARNING **
Notification Type: PROBLEM Service: Puppet staleness Host: deployment-etcd-01 Address: 10.68.19.227 State: WARNING Date/Time: Mon 18 Dec 19:36:22 UTC 2017 Notes URLs: Additional Info: WARNING: 11.11% of data above the warning threshold [3600.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-parsoid09/Puppet staleness is WARNING **
Notification Type: PROBLEM Service: Puppet staleness Host: deployment-parsoid09 Address: 10.68.20.142 State: WARNING Date/Time: Mon 18 Dec 19:33:44 UTC 2017 Notes URLs: Additional Info: WARNING: 10.00% of data above the warning threshold [3600.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-cache-upload04/Puppet staleness is WARNING **
Notification Type: PROBLEM Service: Puppet staleness Host: deployment-cache-upload04 Address: 10.68.18.109 State: WARNING Date/Time: Mon 18 Dec 19:29:50 UTC 2017 Notes URLs: Additional Info: WARNING: 10.00% of data above the warning threshold [3600.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-trending01/Puppet staleness is WARNING **
Notification Type: PROBLEM Service: Puppet staleness Host: deployment-trending01 Address: 10.68.18.186 State: WARNING Date/Time: Mon 18 Dec 19:29:48 UTC 2017 Notes URLs: Additional Info: WARNING: 11.11% of data above the warning threshold [3600.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-sca03/Puppet staleness is WARNING **
Notification Type: PROBLEM Service: Puppet staleness Host: deployment-sca03 Address: 10.68.21.183 State: WARNING Date/Time: Mon 18 Dec 19:29:45 UTC 2017 Notes URLs: Additional Info: WARNING: 11.11% of data above the warning threshold [3600.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** RECOVERY alert - Graphite Labs/Mediawiki Error Rate is OK **
Notification Type: RECOVERY Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: OK Date/Time: Mon 18 Dec 19:28:38 UTC 2017 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [1.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is WARNING **
Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: WARNING Date/Time: Mon 18 Dec 19:23:36 UTC 2017 Notes URLs: Additional Info: WARNING: 20.00% of data above the warning threshold [1.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] beta-code-update-eqiad - Build # 185786 - Failure!
beta-code-update-eqiad - Build # 185786 - Failure: Check console output at https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/185786/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-mx/Puppet errors is CRITICAL **
Notification Type: PROBLEM Service: Puppet errors Host: deployment-mx Address: 10.68.17.78 State: CRITICAL Date/Time: Mon 18 Dec 18:39:21 UTC 2017 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** RECOVERY alert - Graphite Labs/Mediawiki Error Rate is OK **
Notification Type: RECOVERY Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: OK Date/Time: Mon 18 Dec 15:40:08 UTC 2017 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [1.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is WARNING **
Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: WARNING Date/Time: Mon 18 Dec 15:35:08 UTC 2017 Notes URLs: Additional Info: WARNING: 20.00% of data above the warning threshold [1.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-redis02/Puppet errors is CRITICAL **
Notification Type: PROBLEM Service: Puppet errors Host: deployment-redis02 Address: 10.68.16.231 State: CRITICAL Date/Time: Mon 18 Dec 15:33:41 UTC 2017 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-redis01/Puppet errors is CRITICAL **
Notification Type: PROBLEM Service: Puppet errors Host: deployment-redis01 Address: 10.68.16.177 State: CRITICAL Date/Time: Mon 18 Dec 15:27:44 UTC 2017 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** RECOVERY alert - Generic Beta Cluster/English Wikipedia Main page is OK **
Notification Type: RECOVERY Service: English Wikipedia Main page Host: Generic Beta Cluster Address: en.wikipedia.beta.wmflabs.org State: OK Date/Time: Mon 18 Dec 15:23:29 UTC 2017 Notes URLs: Additional Info: HTTP OK: HTTP/1.1 200 OK - 47006 bytes in 0.969 second response time ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - Generic Beta Cluster/English Wikipedia Main page is CRITICAL **
Notification Type: PROBLEM Service: English Wikipedia Main page Host: Generic Beta Cluster Address: en.wikipedia.beta.wmflabs.org State: CRITICAL Date/Time: Mon 18 Dec 15:18:38 UTC 2017 Notes URLs: Additional Info: CRITICAL - Socket timeout after 10 seconds ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-redis06/Puppet errors is CRITICAL **
Notification Type: PROBLEM Service: Puppet errors Host: deployment-redis06 Address: 10.68.20.16 State: CRITICAL Date/Time: Mon 18 Dec 15:15:00 UTC 2017 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
[Betacluster-alerts] ** PROBLEM alert - deployment-trending01/Puppet errors is CRITICAL **
Notification Type: PROBLEM Service: Puppet errors Host: deployment-trending01 Address: 10.68.18.186 State: CRITICAL Date/Time: Mon 18 Dec 11:27:33 UTC 2017 Notes URLs: Additional Info: CRITICAL: 22.22% of data above the critical threshold [0.0] ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/betacluster-alerts
