Notification Type: RECOVERY
Service: Free space - all mounts
Host: deployment-eventlog02
Address: 10.68.18.138
State: OK
Date/Time: Tue 19 Dec 06:55:11 UTC 2017
Notes URLs:
Additional Info:
OK: All targets OK
___
Betacluster-alerts mailing list
Notification Type: RECOVERY
Service: Free space - all mounts
Host: deployment-fluorine02
Address: 10.68.23.106
State: OK
Date/Time: Tue 19 Dec 06:54:00 UTC 2017
Notes URLs:
Additional Info:
OK: All targets OK
___
Betacluster-alerts mailing list
Notification Type: RECOVERY
Service: Free space - all mounts
Host: deployment-kafka03
Address: 10.68.16.138
State: OK
Date/Time: Tue 19 Dec 06:52:27 UTC 2017
Notes URLs:
Additional Info:
OK: All targets OK
___
Betacluster-alerts mailing list
Notification Type: PROBLEM
Service: Puppet staleness
Host: deployment-secureredirexperiment
Address: 10.68.17.132
State: CRITICAL
Date/Time: Tue 19 Dec 06:51:48 UTC 2017
Notes URLs:
Additional Info:
CRITICAL: 10.00% of data above the critical threshold [43200.0]
Notification Type: PROBLEM
Service: Puppet staleness
Host: deployment-ms-be04
Address: 10.68.16.139
State: CRITICAL
Date/Time: Tue 19 Dec 06:49:24 UTC 2017
Notes URLs:
Additional Info:
CRITICAL: 20.00% of data above the critical threshold [43200.0]
Notification Type: PROBLEM
Service: Puppet staleness
Host: deployment-ms-be03
Address: 10.68.22.125
State: CRITICAL
Date/Time: Tue 19 Dec 06:44:37 UTC 2017
Notes URLs:
Additional Info:
CRITICAL: 33.33% of data above the critical threshold [43200.0]
Notification Type: PROBLEM
Service: Puppet staleness
Host: deployment-restbase02
Address: 10.68.17.189
State: CRITICAL
Date/Time: Tue 19 Dec 05:22:25 UTC 2017
Notes URLs:
Additional Info:
CRITICAL: 100.00% of data above the critical threshold [43200.0]
Notification Type: PROBLEM
Service: Puppet staleness
Host: deployment-restbase01
Address: 10.68.16.128
State: CRITICAL
Date/Time: Tue 19 Dec 04:59:27 UTC 2017
Notes URLs:
Additional Info:
CRITICAL: 100.00% of data above the critical threshold [43200.0]
Notification Type: PROBLEM
Service: Mediawiki Error Rate
Host: Graphite Labs
Address: graphite-labs.wikimedia.org
State: WARNING
Date/Time: Tue 19 Dec 04:32:37 UTC 2017
Notes URLs:
Additional Info:
WARNING: 60.00% of data above the warning threshold [1.0]
Notification Type: RECOVERY
Service: Mediawiki Error Rate
Host: Graphite Labs
Address: graphite-labs.wikimedia.org
State: OK
Date/Time: Tue 19 Dec 04:26:37 UTC 2017
Notes URLs:
Additional Info:
OK: Less than 1.00% above the threshold [1.0]
___
Notification Type: PROBLEM
Service: Free space - all mounts
Host: deployment-kafka03
Address: 10.68.16.138
State: WARNING
Date/Time: Tue 19 Dec 04:02:27 UTC 2017
Notes URLs:
Additional Info:
WARNING: deployment-prep.deployment-kafka03.diskspace.root.byte_percentfree
(<66.67%)
Notification Type: RECOVERY
Service: Free space - all mounts
Host: deployment-kafka03
Address: 10.68.16.138
State: OK
Date/Time: Tue 19 Dec 03:31:26 UTC 2017
Notes URLs:
Additional Info:
OK: All targets OK
___
Betacluster-alerts mailing list
Notification Type: PROBLEM
Service: Free space - all mounts
Host: deployment-kafka03
Address: 10.68.16.138
State: WARNING
Date/Time: Tue 19 Dec 03:26:27 UTC 2017
Notes URLs:
Additional Info:
WARNING: deployment-prep.deployment-kafka03.diskspace.root.byte_percentfree
(<11.11%)
Notification Type: PROBLEM
Service: Free space - all mounts
Host: deployment-fluorine02
Address: 10.68.23.106
State: CRITICAL
Date/Time: Tue 19 Dec 03:03:59 UTC 2017
Notes URLs:
Additional Info:
CRITICAL: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree
(<30.00%)
Notification Type: PROBLEM
Service: Puppet errors
Host: deployment-tin
Address: 10.68.21.205
State: CRITICAL
Date/Time: Tue 19 Dec 02:49:20 UTC 2017
Notes URLs:
Additional Info:
CRITICAL: 100.00% of data above the critical threshold [0.0]
___
Notification Type: PROBLEM
Service: Puppet errors
Host: deployment-mira
Address: 10.68.20.135
State: CRITICAL
Date/Time: Tue 19 Dec 02:43:29 UTC 2017
Notes URLs:
Additional Info:
CRITICAL: 100.00% of data above the critical threshold [0.0]
___
Notification Type: PROBLEM
Service: Long lived cherry-picks on puppetmaster
Host: deployment-puppetmaster02
Address: 10.68.21.200
State: CRITICAL
Date/Time: Tue 19 Dec 02:30:44 UTC 2017
Notes URLs:
Additional Info:
CRITICAL: 100.00% of data above the critical threshold [0.0]
Notification Type: PROBLEM
Service: Free space - all mounts
Host: deployment-mx
Address: 10.68.17.78
State: WARNING
Date/Time: Tue 19 Dec 02:04:07 UTC 2017
Notes URLs:
Additional Info:
WARNING: deployment-prep.deployment-mx.diskspace._var_log.byte_percentfree
(<100.00%)
Notification Type: PROBLEM
Service: Free space - all mounts
Host: deployment-eventlog02
Address: 10.68.18.138
State: WARNING
Date/Time: Tue 19 Dec 01:40:11 UTC 2017
Notes URLs:
Additional Info:
WARNING: deployment-prep.deployment-eventlog02.diskspace.root.byte_percentfree
(<33.33%)
Notification Type: PROBLEM
Service: Mediawiki Error Rate
Host: Graphite Labs
Address: graphite-labs.wikimedia.org
State: WARNING
Date/Time: Tue 19 Dec 00:21:37 UTC 2017
Notes URLs:
Additional Info:
WARNING: 20.00% of data above the warning threshold [1.0]
Notification Type: PROBLEM
Service: Mediawiki Error Rate
Host: Graphite Labs
Address: graphite-labs.wikimedia.org
State: CRITICAL
Date/Time: Tue 19 Dec 00:16:36 UTC 2017
Notes URLs:
Additional Info:
CRITICAL: 20.00% of data above the critical threshold [10.0]
Notification Type: PROBLEM
Service: Free space - all mounts
Host: deployment-fluorine02
Address: 10.68.23.106
State: WARNING
Date/Time: Tue 19 Dec 00:09:00 UTC 2017
Notes URLs:
Additional Info:
WARNING: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree
(<40.00%)
Notification Type: PROBLEM
Service: Puppet errors
Host: deployment-kafka-jumbo-1
Address: 10.68.23.243
State: CRITICAL
Date/Time: Mon 18 Dec 23:37:24 UTC 2017
Notes URLs:
Additional Info:
CRITICAL: 100.00% of data above the critical threshold [0.0]
Notification Type: PROBLEM
Service: Puppet errors
Host: deployment-netbox
Address: 10.68.19.203
State: CRITICAL
Date/Time: Mon 18 Dec 23:36:09 UTC 2017
Notes URLs:
Additional Info:
CRITICAL: 100.00% of data above the critical threshold [0.0]
___
Notification Type: PROBLEM
Service: Puppet errors
Host: deployment-kafka-jumbo-2
Address: 10.68.16.87
State: CRITICAL
Date/Time: Mon 18 Dec 23:30:56 UTC 2017
Notes URLs:
Additional Info:
CRITICAL: 100.00% of data above the critical threshold [0.0]
Notification Type: PROBLEM
Service: Puppet errors
Host: deployment-etcd-01
Address: 10.68.19.227
State: CRITICAL
Date/Time: Mon 18 Dec 22:35:42 UTC 2017
Notes URLs:
Additional Info:
CRITICAL: 100.00% of data above the critical threshold [0.0]
___
Notification Type: PROBLEM
Service: Puppet errors
Host: deployment-sentry01
Address: 10.68.19.148
State: CRITICAL
Date/Time: Mon 18 Dec 22:20:13 UTC 2017
Notes URLs:
Additional Info:
CRITICAL: 100.00% of data above the critical threshold [0.0]
___
Notification Type: PROBLEM
Service: App Server Main HTTP Response
Host: deployment-mediawiki05
Address: 10.68.22.21
State: CRITICAL
Date/Time: Mon 18 Dec 21:31:31 UTC 2017
Notes URLs:
Additional Info:
HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 23217 bytes in 0.579 second
response
Notification Type: PROBLEM
Service: Mediawiki Error Rate
Host: Graphite Labs
Address: graphite-labs.wikimedia.org
State: WARNING
Date/Time: Mon 18 Dec 21:31:36 UTC 2017
Notes URLs:
Additional Info:
WARNING: 60.00% of data above the warning threshold [1.0]
Notification Type: PROBLEM
Service: English Wikipedia Mobile Main page
Host: Generic Beta Cluster
Address: en.wikipedia.beta.wmflabs.org
State: CRITICAL
Date/Time: Mon 18 Dec 21:31:15 UTC 2017
Notes URLs:
Additional Info:
HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 19229 bytes in
Notification Type: PROBLEM
Service: App Server Main HTTP Response
Host: deployment-mediawiki04
Address: 10.68.19.128
State: CRITICAL
Date/Time: Mon 18 Dec 21:30:34 UTC 2017
Notes URLs:
Additional Info:
HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 23217 bytes in 0.745 second
response
Notification Type: PROBLEM
Service: App Server Main HTTP Response
Host: deployment-mediawiki06
Address: 10.68.19.241
State: CRITICAL
Date/Time: Mon 18 Dec 21:28:49 UTC 2017
Notes URLs:
Additional Info:
HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 23217 bytes in 0.670 second
response
Notification Type: PROBLEM
Service: English Wikipedia Main page
Host: Generic Beta Cluster
Address: en.wikipedia.beta.wmflabs.org
State: CRITICAL
Date/Time: Mon 18 Dec 21:28:43 UTC 2017
Notes URLs:
Additional Info:
HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 23770 bytes in 0.749
Notification Type: RECOVERY
Service: Mediawiki Error Rate
Host: Graphite Labs
Address: graphite-labs.wikimedia.org
State: OK
Date/Time: Mon 18 Dec 20:54:37 UTC 2017
Notes URLs:
Additional Info:
OK: Less than 1.00% above the threshold [1.0]
___
Notification Type: PROBLEM
Service: Mediawiki Error Rate
Host: Graphite Labs
Address: graphite-labs.wikimedia.org
State: WARNING
Date/Time: Mon 18 Dec 20:39:36 UTC 2017
Notes URLs:
Additional Info:
WARNING: 80.00% of data above the warning threshold [1.0]
Notification Type: PROBLEM
Service: Puppet errors
Host: deployment-eventlogging04
Address: 10.68.23.204
State: CRITICAL
Date/Time: Mon 18 Dec 20:34:28 UTC 2017
Notes URLs:
Additional Info:
CRITICAL: 100.00% of data above the critical threshold [0.0]
Notification Type: RECOVERY
Service: Puppet staleness
Host: deployment-sentry01
Address: 10.68.19.148
State: OK
Date/Time: Mon 18 Dec 19:56:29 UTC 2017
Notes URLs:
Additional Info:
OK: Less than 1.00% above the threshold [3600.0]
___
Notification Type: PROBLEM
Service: Puppet staleness
Host: deployment-secureredirexperiment
Address: 10.68.17.132
State: WARNING
Date/Time: Mon 18 Dec 19:56:48 UTC 2017
Notes URLs:
Additional Info:
WARNING: 60.00% of data above the warning threshold [3600.0]
Notification Type: RECOVERY
Service: Puppet staleness
Host: deployment-tmh01
Address: 10.68.16.211
State: OK
Date/Time: Mon 18 Dec 19:54:11 UTC 2017
Notes URLs:
Additional Info:
OK: Less than 1.00% above the threshold [3600.0]
___
Notification Type: RECOVERY
Service: Puppet staleness
Host: deployment-db04
Address: 10.68.18.35
State: OK
Date/Time: Mon 18 Dec 19:50:41 UTC 2017
Notes URLs:
Additional Info:
OK: Less than 1.00% above the threshold [3600.0]
___
Betacluster-alerts
Notification Type: PROBLEM
Service: Mediawiki Error Rate
Host: Graphite Labs
Address: graphite-labs.wikimedia.org
State: CRITICAL
Date/Time: Mon 18 Dec 19:49:36 UTC 2017
Notes URLs:
Additional Info:
CRITICAL: 20.00% of data above the critical threshold [10.0]
Notification Type: PROBLEM
Service: Puppet staleness
Host: deployment-ms-be04
Address: 10.68.16.139
State: WARNING
Date/Time: Mon 18 Dec 19:49:24 UTC 2017
Notes URLs:
Additional Info:
WARNING: 20.00% of data above the warning threshold [3600.0]
___
Notification Type: RECOVERY
Service: Puppet staleness
Host: deployment-memc06
Address: 10.68.22.239
State: OK
Date/Time: Mon 18 Dec 19:48:45 UTC 2017
Notes URLs:
Additional Info:
OK: Less than 1.00% above the threshold [3600.0]
___
Notification Type: RECOVERY
Service: Puppet staleness
Host: deployment-cpjobqueue
Address: 10.68.22.161
State: OK
Date/Time: Mon 18 Dec 19:46:58 UTC 2017
Notes URLs:
Additional Info:
OK: Less than 1.00% above the threshold [3600.0]
___
Notification Type: RECOVERY
Service: Puppet staleness
Host: deployment-elastic05
Address: 10.68.20.21
State: OK
Date/Time: Mon 18 Dec 19:46:49 UTC 2017
Notes URLs:
Additional Info:
OK: Less than 1.00% above the threshold [3600.0]
___
Notification Type: RECOVERY
Service: Puppet staleness
Host: deployment-etcd-01
Address: 10.68.19.227
State: OK
Date/Time: Mon 18 Dec 19:46:22 UTC 2017
Notes URLs:
Additional Info:
OK: Less than 1.00% above the threshold [3600.0]
___
Notification Type: RECOVERY
Service: Puppet staleness
Host: deployment-aqs01
Address: 10.68.18.237
State: OK
Date/Time: Mon 18 Dec 19:46:13 UTC 2017
Notes URLs:
Additional Info:
OK: Less than 1.00% above the threshold [3600.0]
___
Notification Type: PROBLEM
Service: Puppet staleness
Host: deployment-ms-be03
Address: 10.68.22.125
State: WARNING
Date/Time: Mon 18 Dec 19:44:37 UTC 2017
Notes URLs:
Additional Info:
WARNING: 33.33% of data above the warning threshold [3600.0]
___
Notification Type: PROBLEM
Service: Puppet staleness
Host: deployment-tmh01
Address: 10.68.16.211
State: WARNING
Date/Time: Mon 18 Dec 19:44:09 UTC 2017
Notes URLs:
Additional Info:
WARNING: 12.50% of data above the warning threshold [3600.0]
___
beta-code-update-eqiad - Build # 185792 - Fixed:
Check console output at https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/185792/ to view the results.___
Betacluster-alerts mailing list
Betacluster-alerts@lists.wikimedia.org
Notification Type: RECOVERY
Service: Puppet staleness
Host: deployment-parsoid09
Address: 10.68.20.142
State: OK
Date/Time: Mon 18 Dec 19:43:44 UTC 2017
Notes URLs:
Additional Info:
OK: Less than 1.00% above the threshold [3600.0]
___
Notification Type: PROBLEM
Service: Puppet staleness
Host: deployment-elastic05
Address: 10.68.20.21
State: WARNING
Date/Time: Mon 18 Dec 19:41:49 UTC 2017
Notes URLs:
Additional Info:
WARNING: 11.11% of data above the warning threshold [3600.0]
Notification Type: PROBLEM
Service: Puppet staleness
Host: deployment-aqs01
Address: 10.68.18.237
State: WARNING
Date/Time: Mon 18 Dec 19:41:12 UTC 2017
Notes URLs:
Additional Info:
WARNING: 11.11% of data above the warning threshold [3600.0]
___
Notification Type: PROBLEM
Service: Puppet staleness
Host: deployment-db04
Address: 10.68.18.35
State: WARNING
Date/Time: Mon 18 Dec 19:40:42 UTC 2017
Notes URLs:
Additional Info:
WARNING: 10.00% of data above the warning threshold [3600.0]
___
Notification Type: RECOVERY
Service: Puppet staleness
Host: deployment-cache-upload04
Address: 10.68.18.109
State: OK
Date/Time: Mon 18 Dec 19:39:50 UTC 2017
Notes URLs:
Additional Info:
OK: Less than 1.00% above the threshold [3600.0]
___
Notification Type: RECOVERY
Service: Puppet staleness
Host: deployment-trending01
Address: 10.68.18.186
State: OK
Date/Time: Mon 18 Dec 19:39:48 UTC 2017
Notes URLs:
Additional Info:
OK: Less than 1.00% above the threshold [3600.0]
___
Notification Type: RECOVERY
Service: Puppet staleness
Host: deployment-sca03
Address: 10.68.21.183
State: OK
Date/Time: Mon 18 Dec 19:39:45 UTC 2017
Notes URLs:
Additional Info:
OK: Less than 1.00% above the threshold [3600.0]
___
Notification Type: PROBLEM
Service: Puppet staleness
Host: deployment-memc06
Address: 10.68.22.239
State: WARNING
Date/Time: Mon 18 Dec 19:38:47 UTC 2017
Notes URLs:
Additional Info:
WARNING: 10.00% of data above the warning threshold [3600.0]
___
Notification Type: PROBLEM
Service: Puppet staleness
Host: deployment-cpjobqueue
Address: 10.68.22.161
State: WARNING
Date/Time: Mon 18 Dec 19:36:57 UTC 2017
Notes URLs:
Additional Info:
WARNING: 10.00% of data above the warning threshold [3600.0]
Notification Type: PROBLEM
Service: Puppet staleness
Host: deployment-etcd-01
Address: 10.68.19.227
State: WARNING
Date/Time: Mon 18 Dec 19:36:22 UTC 2017
Notes URLs:
Additional Info:
WARNING: 11.11% of data above the warning threshold [3600.0]
___
Notification Type: PROBLEM
Service: Puppet staleness
Host: deployment-parsoid09
Address: 10.68.20.142
State: WARNING
Date/Time: Mon 18 Dec 19:33:44 UTC 2017
Notes URLs:
Additional Info:
WARNING: 10.00% of data above the warning threshold [3600.0]
Notification Type: PROBLEM
Service: Puppet staleness
Host: deployment-cache-upload04
Address: 10.68.18.109
State: WARNING
Date/Time: Mon 18 Dec 19:29:50 UTC 2017
Notes URLs:
Additional Info:
WARNING: 10.00% of data above the warning threshold [3600.0]
Notification Type: PROBLEM
Service: Puppet staleness
Host: deployment-trending01
Address: 10.68.18.186
State: WARNING
Date/Time: Mon 18 Dec 19:29:48 UTC 2017
Notes URLs:
Additional Info:
WARNING: 11.11% of data above the warning threshold [3600.0]
Notification Type: PROBLEM
Service: Puppet staleness
Host: deployment-sca03
Address: 10.68.21.183
State: WARNING
Date/Time: Mon 18 Dec 19:29:45 UTC 2017
Notes URLs:
Additional Info:
WARNING: 11.11% of data above the warning threshold [3600.0]
___
Notification Type: RECOVERY
Service: Mediawiki Error Rate
Host: Graphite Labs
Address: graphite-labs.wikimedia.org
State: OK
Date/Time: Mon 18 Dec 19:28:38 UTC 2017
Notes URLs:
Additional Info:
OK: Less than 1.00% above the threshold [1.0]
___
Notification Type: PROBLEM
Service: Mediawiki Error Rate
Host: Graphite Labs
Address: graphite-labs.wikimedia.org
State: WARNING
Date/Time: Mon 18 Dec 19:23:36 UTC 2017
Notes URLs:
Additional Info:
WARNING: 20.00% of data above the warning threshold [1.0]
beta-code-update-eqiad - Build # 185786 - Failure:
Check console output at https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/185786/ to view the results.___
Betacluster-alerts mailing list
Betacluster-alerts@lists.wikimedia.org
Notification Type: PROBLEM
Service: Puppet errors
Host: deployment-mx
Address: 10.68.17.78
State: CRITICAL
Date/Time: Mon 18 Dec 18:39:21 UTC 2017
Notes URLs:
Additional Info:
CRITICAL: 100.00% of data above the critical threshold [0.0]
___
Notification Type: RECOVERY
Service: Mediawiki Error Rate
Host: Graphite Labs
Address: graphite-labs.wikimedia.org
State: OK
Date/Time: Mon 18 Dec 15:40:08 UTC 2017
Notes URLs:
Additional Info:
OK: Less than 1.00% above the threshold [1.0]
___
Notification Type: PROBLEM
Service: Mediawiki Error Rate
Host: Graphite Labs
Address: graphite-labs.wikimedia.org
State: WARNING
Date/Time: Mon 18 Dec 15:35:08 UTC 2017
Notes URLs:
Additional Info:
WARNING: 20.00% of data above the warning threshold [1.0]
Notification Type: PROBLEM
Service: Puppet errors
Host: deployment-redis02
Address: 10.68.16.231
State: CRITICAL
Date/Time: Mon 18 Dec 15:33:41 UTC 2017
Notes URLs:
Additional Info:
CRITICAL: 100.00% of data above the critical threshold [0.0]
___
Notification Type: PROBLEM
Service: Puppet errors
Host: deployment-redis01
Address: 10.68.16.177
State: CRITICAL
Date/Time: Mon 18 Dec 15:27:44 UTC 2017
Notes URLs:
Additional Info:
CRITICAL: 100.00% of data above the critical threshold [0.0]
___
Notification Type: RECOVERY
Service: English Wikipedia Main page
Host: Generic Beta Cluster
Address: en.wikipedia.beta.wmflabs.org
State: OK
Date/Time: Mon 18 Dec 15:23:29 UTC 2017
Notes URLs:
Additional Info:
HTTP OK: HTTP/1.1 200 OK - 47006 bytes in 0.969 second response time
Notification Type: PROBLEM
Service: English Wikipedia Main page
Host: Generic Beta Cluster
Address: en.wikipedia.beta.wmflabs.org
State: CRITICAL
Date/Time: Mon 18 Dec 15:18:38 UTC 2017
Notes URLs:
Additional Info:
CRITICAL - Socket timeout after 10 seconds
Notification Type: PROBLEM
Service: Puppet errors
Host: deployment-redis06
Address: 10.68.20.16
State: CRITICAL
Date/Time: Mon 18 Dec 15:15:00 UTC 2017
Notes URLs:
Additional Info:
CRITICAL: 100.00% of data above the critical threshold [0.0]
___
Notification Type: PROBLEM
Service: Puppet errors
Host: deployment-trending01
Address: 10.68.18.186
State: CRITICAL
Date/Time: Mon 18 Dec 11:27:33 UTC 2017
Notes URLs:
Additional Info:
CRITICAL: 22.22% of data above the critical threshold [0.0]
___
76 matches
Mail list logo
