Giuseppe Lavagetto has uploaded a new change for review.
https://gerrit.wikimedia.org/r/312205
Change subject: hieradata: stop repeating data for clusters
......................................................................
hieradata: stop repeating data for clusters
Basically every time we want to add a new cluster, at the moment we need
to add it in two places in hiera at least. With this change we remove
the need to do it, and unify everything into one single data structure
Change-Id: If5a4c8c3d8bc5b8656900a81a06935037823a9e3
---
M hieradata/common/monitoring.yaml
M manifests/role/icinga.pp
A modules/monitoring/manifests/cluster_groups.pp
3 files changed, 28 insertions(+), 220 deletions(-)
git pull ssh://gerrit.wikimedia.org:29418/operations/puppet
refs/changes/05/312205/1
diff --git a/hieradata/common/monitoring.yaml b/hieradata/common/monitoring.yaml
index 54b1207..10f65ef 100644
--- a/hieradata/common/monitoring.yaml
+++ b/hieradata/common/monitoring.yaml
@@ -2,225 +2,6 @@
# This is a yaml file. Remember to keep indentation otherwise the structure
# will not be what you intend
groups:
- analytics_eqiad:
- description: analytics servers in eqiad
-
-# Analytics Query Service
- aqs_eqiad:
- description: Analytics Query Service eqiad
- aqs_codfw:
- description: Analytics Query Service codfw
-
-# Cache
- cache_text_codfw:
- description: codfw text Varnish
- cache_text_eqiad:
- description: eqiad text Varnish
- cache_text_esams:
- description: esams text Varnish
- cache_text_ulsfo:
- description: ulsfo text Varnish
- cache_upload_codfw:
- description: codfw upload Varnish
- cache_upload_eqiad:
- description: eqiad upload Varnish
- cache_upload_esams:
- description: esams upload Varnish
- cache_upload_ulsfo:
- description: ulsfo upload Varnish
- cache_misc_codfw:
- description: Misc caches codfw
- cache_misc_eqiad:
- description: Misc caches eqiad
- cache_misc_esams:
- description: Misc caches esams
- cache_misc_ulsfo:
- description: Misc caches ulsfo
- cache_maps_codfw:
- description: Maps caches codfw
- cache_maps_eqiad:
- description: Maps caches eqiad
- cache_maps_esams:
- description: Maps caches esams
- cache_maps_ulsfo:
- description: Maps caches ulsfo
-
-# Elasticsearch
- elasticsearch_eqiad:
- description: eqiad elasticsearch servers
- elasticsearch_codfw:
- description: codfw elasticsearch servers
- elasticsearch_esams:
- description: esams elasticsearch servers
- elasticsearch_ulsfo:
- description: ulsfo elasticsearch servers
- relforge_eqiad:
- description: eqiad elasticsearch relforge servers
-
-# Etcd
- etcd_eqiad:
- description: eqiad Etcd
-
- # Eventbus
- eventbus_eqiad:
- description: eventbus eqiad servers
-
- eventbus_codfw:
- description: eventbus codfw servers
-
-# Ganeti
- ganeti_eqiad:
- description: Ganeti virt cluster eqiad
- ganeti_codfw:
- description: Ganeti virt cluster codfw
-
-# Labs OpenStack Nova (labvirt***)
- labvirt_eqiad:
- description: eqiad virt servers
- labvirt_codfw:
- description: codfw virt servers
-
-# Labs Services
- labs_eqiad:
- description: eqiad labs services
- labs_codfw:
- description: codfw labs services
-
-# LVS
- lvs:
- description: LVS
- lvs_eqiad:
- description: eqiad LVS servers
- lvs_codfw:
- description: codfw LVS servers
- lvs_ulsfo:
- description: ulsfo LVS servers
- lvs_esams:
- description: esams LVS servers
-
-# Logstash
- logstash_eqiad:
- description: eqiad logstash
-
-# Maps
- maps_eqiad:
- description: eqiad maps servers
- maps_codfw:
- description: codfw maps servers
-
-# MediaWiki
- appserver_eqiad:
- description: eqiad application servers
- api_appserver_eqiad:
- description: eqiad API application servers
- imagescaler_eqiad:
- description: eqiad image scalers
- jobrunner_eqiad:
- description: eqiad jobrunner application servers
- videoscaler_eqiad:
- description: eqiad video scaler
-
- appserver_codfw:
- description: codfw application servers
- api_appserver_codfw:
- description: codfw API application servers
- imagescaler_codfw:
- description: codfw image scalers
- jobrunner_codfw:
- description: codfw jobrunner application servers
- videoscaler_codfw:
- description: codfw video scaler
-
-# Memcached
- memcached_eqiad:
- description: eqiad memcached
- memcached_codfw:
- description: codfw memcached
-
-# MySQL
- mysql_eqiad:
- description: eqiad mysql core
- mysql_codfw:
- description: codfw mysql core
-
-# OCG
- ocg_eqiad:
- description: offline content generator eqiad
-
-# Puppet
- puppet_eqiad:
- description: Puppetmasters eqiad
- puppet_codfw:
- description: Puppetmasters codfw
-
-# Parsoid
- parsoid_eqiad:
- description: eqiad parsoid servers
- parsoid_codfw:
- description: codfw parsoid servers
-
-# RCStream
- rcstream_eqiad:
- description: eqiad rcstream
-
-# RedisDB
- redis_eqiad:
- description: eqiad Redis
- redis_codfw:
- description: codfw Redis
-
-# RESTBase
- restbase_eqiad:
- description: Restbase eqiad
- restbase_codfw:
- description: Restbase codfw
- restbase_test_eqiad:
- description: Restbase test/staging eqiad
- restbase_test_codfw:
- description: Restbase test/staging codfw
-
-# Service Clusters
- sca_eqiad:
- description: Service Cluster A servers eqiad
- sca_codfw:
- description: Service Cluster A servers codfw
- scb_eqiad:
- description: Service Cluster B servers eqiad
- scb_codfw:
- description: Service Cluster B servers codfw
-
-# Swift
- swift:
- description: swift servers
-
-# Wikidata Query Service
- wdqs_eqiad:
- description: Wikidata Query Service (eqiad)
- wdqs_codfw:
- description: Wikidata Query Service (codfw)
-
-# Zotero
- zotero_eqiad:
- description: Zotero eqiad
- zotero_codfw:
- description: Zotero codfw
-
-# Labs
- labsnfs_eqiad:
- description: eqiad labsnfs server servers
- labsnfs_codfw:
- description: codfw labsnfs server servers
-
-# Misc
- misc_eqiad:
- description: eqiad misc servers
- misc_codfw:
- description: codfw misc servers
- misc_esams:
- description: esams misc servers
- misc_ulsfo:
- description: ulsfo misc servers
-
# Facilities
pdus:
description: PDUs
diff --git a/manifests/role/icinga.pp b/manifests/role/icinga.pp
index c40bf8a..f75971a 100644
--- a/manifests/role/icinga.pp
+++ b/manifests/role/icinga.pp
@@ -10,7 +10,7 @@
#
class role::icinga(
$ircbot = true,
-){
+ ){
include facilities
include icinga::monitor::checkpaging
include icinga::nsca::firewall
@@ -34,6 +34,11 @@
include standard
include base::firewall
+ # Create groups for puppet-defined clusters
+ $cluster_sites = hiera('ganglia_clusters')
+ create_resources(monitoring::cluster_groups, $cluster_sites)
+
+ # Groups that have no correspondence in puppet
$monitoring_groups = hiera('monitoring::groups')
create_resources(monitoring::group, $monitoring_groups)
diff --git a/modules/monitoring/manifests/cluster_groups.pp
b/modules/monitoring/manifests/cluster_groups.pp
new file mode 100644
index 0000000..712f2a5
--- /dev/null
+++ b/modules/monitoring/manifests/cluster_groups.pp
@@ -0,0 +1,22 @@
+# === Define monitoring::cluster_groups
+#
+# Wrapper define used for deriving monitoring::group resources
+# from the ganglia_clusters hiera data structure.
+#
+# This can probably go away once we have the future parser
+define monitoring::cluster_groups($description, $id, $sites) {
+
+ $site_names = keys($sites)
+
+ # From the data we get from ganglia_clusters, let's create
+ # an hash like
+ # appserver_eqiad => {description => 'Application servers eqiad'}
+ $cluster_groups =hash(
+ split(
+ inline_template(
+ '<%= @site_names.map{ |s| ["#{@title}_#{s}", {"description" =>
"#{@description} #{s}"}] }.flatten.join("|||")'
+ ), '\|\|\|'
+ )
+ )
+ create_resources(monitoring::group, $cluster_groups)
+}
--
To view, visit https://gerrit.wikimedia.org/r/312205
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: If5a4c8c3d8bc5b8656900a81a06935037823a9e3
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto <glavage...@wikimedia.org>
_______________________________________________
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
