Giuseppe Lavagetto has uploaded a new change for review.
https://gerrit.wikimedia.org/r/199841
Change subject: icinga: check its own configuration
......................................................................
icinga: check its own configuration
At the moment, whenever one puppet change breaks the icinga config we
get exactly one puppet failure from neon, which then recovers (as the
icinga config will be reloaded just once), but it seems we can't notice
it promptly. The net effect is that the icinga config is often broken
and it requires me and others to fix it a posteriori when we're doing
another change.
Introducting an icinga check on its own configuration should allow us to
notice the problem more explicitly, and persistently.
Change-Id: I166e1113478e64f9763702f9b391e78027edcafe
---
M modules/icinga/manifests/init.pp
A modules/nagios_common/files/check_commands/check_icinga_config
M modules/nagios_common/manifests/check_command.pp
M modules/nagios_common/manifests/commands.pp
A modules/nagios_common/templates/check_icinga_config.cfg.erb
5 files changed, 33 insertions(+), 1 deletion(-)
git pull ssh://gerrit.wikimedia.org:29418/operations/puppet
refs/changes/41/199841/1
diff --git a/modules/icinga/manifests/init.pp b/modules/icinga/manifests/init.pp
index 7beb4b8..780cd6d 100644
--- a/modules/icinga/manifests/init.pp
+++ b/modules/icinga/manifests/init.pp
@@ -155,4 +155,10 @@
group => 'www-data',
mode => '0664',
}
+
+ montioring::service { 'check_icinga_config':
+ description => 'Check correctness of the icinga
configuration',
+ check_command => 'check_icinga_config',
+ normal_check_interval => 10,
+ }
}
diff --git a/modules/nagios_common/files/check_commands/check_icinga_config
b/modules/nagios_common/files/check_commands/check_icinga_config
new file mode 100755
index 0000000..208240b
--- /dev/null
+++ b/modules/nagios_common/files/check_commands/check_icinga_config
@@ -0,0 +1,11 @@
+#!/bin/bash
+set -u
+CONFIG_FILE=${1}
+/usr/sbin/icinga -v ${CONFIG_FILE} > /dev/null 2>&1
+RETVAL=$?
+if [ ${RETVAL} -eq 0 ]; then
+ echo "Icinga configuration is correct"
+ exit 0
+fi;
+echo "Icinga configuration contains errors, please check!"
+exit 2
diff --git a/modules/nagios_common/manifests/check_command.pp
b/modules/nagios_common/manifests/check_command.pp
index d2ce446..f3f9013 100644
--- a/modules/nagios_common/manifests/check_command.pp
+++ b/modules/nagios_common/manifests/check_command.pp
@@ -82,7 +82,7 @@
# Path to source for the plugin configuration.
# Defaults to puppet:///modules/nagios_common/check_commands/$title.cfg
#
-# [*source*]
+# [*content*]
# String content to use for the plugin configuration.
# Should not be used with the source parameter.
#
diff --git a/modules/nagios_common/manifests/commands.pp
b/modules/nagios_common/manifests/commands.pp
index d353e58..c8264b7 100644
--- a/modules/nagios_common/manifests/commands.pp
+++ b/modules/nagios_common/manifests/commands.pp
@@ -93,6 +93,16 @@
group => $group,
}
+ # Check that the icinga config works
+ nagios_common::check_command { 'check_icinga_config':
+ ensure => present,
+ config_content =>
template('nagios_common/check_icinga_config.cfg.erb'),
+ config_dir => $config_dir,
+ owner => $owner,
+ group => $group,
+ require => File["${config_dir}/commands"],
+ }
+
file { "${config_dir}/checkcommands.cfg":
source => 'puppet:///modules/nagios_common/checkcommands.cfg',
owner => $owner,
diff --git a/modules/nagios_common/templates/check_icinga_config.cfg.erb
b/modules/nagios_common/templates/check_icinga_config.cfg.erb
new file mode 100644
index 0000000..ee76cae
--- /dev/null
+++ b/modules/nagios_common/templates/check_icinga_config.cfg.erb
@@ -0,0 +1,5 @@
+# Check icinga config command definition
+define command{
+ command_name check_icinga_config
+ command_line /usr/lib/nagios/plugins/<%= @title -%> <%= @config_dir
-%>/icinga.cfg
+}
--
To view, visit https://gerrit.wikimedia.org/r/199841
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I166e1113478e64f9763702f9b391e78027edcafe
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto <[email protected]>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
