[MediaWiki-commits] [Gerrit] librenms: add librenms::syslog class - change (operations/puppet)
Faidon Liambotis has uploaded a new change for review. https://gerrit.wikimedia.org/r/106780 Change subject: librenms: add librenms::syslog class .. librenms: add librenms::syslog class Add a librenms::syslog class which sets up a syslog listener that pipes to LibreNMS. This sets up a separate rsyslog instance that receives messages in syslog (UDP 514) and pipes them to syslog.php. Using a separate instance seems more complicated at first but provides certain important benefits: * The separate instance runs as the librenms user and hence is able to read the configuration file without giving access to the whole syslog group. * There's no mixing of system syslog with the remote syslog and no messy filtering to avoid logging the local system's logs to LibreNMS. * The received loglines are only stored in LibreNMS and are not forwarded to the rest of the syslog config (local log files, remote syslog servers etc.). While at it, also add some default options to ignore spammy Juniper lines from the logs. Change-Id: I7428047f04ae690af807f211218537c8e79628d0 --- M manifests/role/librenms.pp A modules/librenms/files/rsyslog-upstart.conf A modules/librenms/files/rsyslog.conf M modules/librenms/manifests/init.pp A modules/librenms/manifests/syslog.pp 5 files changed, 91 insertions(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/80/106780/1 diff --git a/manifests/role/librenms.pp b/manifests/role/librenms.pp index 92687ae..473d648 100644 --- a/manifests/role/librenms.pp +++ b/manifests/role/librenms.pp @@ -44,7 +44,6 @@ }, 'enable_inventory' = 1, -'enable_syslog'= 1, 'email_backend'= 'sendmail', 'alerts' = { 'port_util_alert' = true, @@ -58,6 +57,14 @@ }, }, +'enable_syslog'= 1, +'syslog_filter'= [ +'message repeated', +'Connection from UDP: [', +'CMD ( /usr/libexec/atrun)', +'CMD (newsyslog)', +], + 'auth_mechanism' = 'mysql', } @@ -65,6 +72,9 @@ install_dir = $install_dir, config = $config, } +class { '::librenms::syslog': +require = Class['::librenms'] +} install_certificate { $sitename: } diff --git a/modules/librenms/files/rsyslog-upstart.conf b/modules/librenms/files/rsyslog-upstart.conf new file mode 100644 index 000..500abf9 --- /dev/null +++ b/modules/librenms/files/rsyslog-upstart.conf @@ -0,0 +1,13 @@ +# This file is managed by Puppet, librenms module + +description LibreNMS syslog daemon + +start on filesystem +stop on runlevel [06] + +expect fork +respawn + +script +exec rsyslogd -c5 -f /etc/librenms-rsyslog.conf -i /var/run/librenms-rsyslogd.pid +end script diff --git a/modules/librenms/files/rsyslog.conf b/modules/librenms/files/rsyslog.conf new file mode 100644 index 000..7db0178 --- /dev/null +++ b/modules/librenms/files/rsyslog.conf @@ -0,0 +1,17 @@ +$ModLoad imudp +$UDPServerRun 514 + +$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat +$PreserveFQDN on +$RepeatedMsgReduction on + +$PrivDropToUser librenms +$PrivDropToGroup librenms + +# perform rsyslog magic instead of plain programname +# Juniper emits e.g. /kernel: as the tag, which makes programname +# write our own parser instead, that doesn't consider / as a terminator +$template librenms,%fromhost%||%syslogfacility-text%||%syslogpriority-text%||%syslogseverity-text%||%syslogtag%||%$year%-%$month%-%$day% %timereported:8:25%||%msg%||%syslogtag:R,ERE,1:([^:\[]+)(\[.*|:.*)?--end%\n +$ModLoad omprog +$ActionOMProgBinary /usr/local/sbin/librenms-syslog +*.* :omprog:;librenms diff --git a/modules/librenms/manifests/init.pp b/modules/librenms/manifests/init.pp index 07c3e11..24c3364 100644 --- a/modules/librenms/manifests/init.pp +++ b/modules/librenms/manifests/init.pp @@ -96,4 +96,9 @@ minute = '*/5', require = User['librenms'], } + +file { '/usr/local/sbin/librenms-syslog': +ensure = link, +target = ${install_dir}/syslog.php, +} } diff --git a/modules/librenms/manifests/syslog.pp b/modules/librenms/manifests/syslog.pp new file mode 100644 index 000..197a4da --- /dev/null +++ b/modules/librenms/manifests/syslog.pp @@ -0,0 +1,45 @@ +# == Class: librenms::syslog +# +# Sets up a separate rsyslog instance that receives messages in syslog (UDP +# 514) and forwards them to librenms' syslog script. +# +# Using a separate instance seems more complicated at first but provides +# certain important benefits: +# * The separate instance runs as the librenms user and hence is able to read +# the configuration file without giving access to the whole syslog group +# * There's no mixing of system syslog with the remote syslog and no messy +# filtering to avoid logging the local system's logs to
[MediaWiki-commits] [Gerrit] librenms: add librenms::syslog class - change (operations/puppet)
Faidon Liambotis has submitted this change and it was merged. Change subject: librenms: add librenms::syslog class .. librenms: add librenms::syslog class Add a librenms::syslog class which sets up a syslog listener that pipes to LibreNMS. This sets up a separate rsyslog instance that receives messages in syslog (UDP 514) and pipes them to syslog.php. Using a separate instance seems more complicated at first but provides certain important benefits: * The separate instance runs as the librenms user and hence is able to read the configuration file without giving access to the whole syslog group. * There's no mixing of system syslog with the remote syslog and no messy filtering to avoid logging the local system's logs to LibreNMS. * The received loglines are only stored in LibreNMS and are not forwarded to the rest of the syslog config (local log files, remote syslog servers etc.). While at it, also add some default options to ignore spammy Juniper lines from the logs. Change-Id: I7428047f04ae690af807f211218537c8e79628d0 --- M manifests/role/librenms.pp A modules/librenms/files/rsyslog-upstart.conf A modules/librenms/files/rsyslog.conf M modules/librenms/manifests/init.pp A modules/librenms/manifests/syslog.pp 5 files changed, 91 insertions(+), 1 deletion(-) Approvals: Faidon Liambotis: Looks good to me, approved jenkins-bot: Verified diff --git a/manifests/role/librenms.pp b/manifests/role/librenms.pp index 92687ae..473d648 100644 --- a/manifests/role/librenms.pp +++ b/manifests/role/librenms.pp @@ -44,7 +44,6 @@ }, 'enable_inventory' = 1, -'enable_syslog'= 1, 'email_backend'= 'sendmail', 'alerts' = { 'port_util_alert' = true, @@ -58,6 +57,14 @@ }, }, +'enable_syslog'= 1, +'syslog_filter'= [ +'message repeated', +'Connection from UDP: [', +'CMD ( /usr/libexec/atrun)', +'CMD (newsyslog)', +], + 'auth_mechanism' = 'mysql', } @@ -65,6 +72,9 @@ install_dir = $install_dir, config = $config, } +class { '::librenms::syslog': +require = Class['::librenms'] +} install_certificate { $sitename: } diff --git a/modules/librenms/files/rsyslog-upstart.conf b/modules/librenms/files/rsyslog-upstart.conf new file mode 100644 index 000..500abf9 --- /dev/null +++ b/modules/librenms/files/rsyslog-upstart.conf @@ -0,0 +1,13 @@ +# This file is managed by Puppet, librenms module + +description LibreNMS syslog daemon + +start on filesystem +stop on runlevel [06] + +expect fork +respawn + +script +exec rsyslogd -c5 -f /etc/librenms-rsyslog.conf -i /var/run/librenms-rsyslogd.pid +end script diff --git a/modules/librenms/files/rsyslog.conf b/modules/librenms/files/rsyslog.conf new file mode 100644 index 000..7db0178 --- /dev/null +++ b/modules/librenms/files/rsyslog.conf @@ -0,0 +1,17 @@ +$ModLoad imudp +$UDPServerRun 514 + +$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat +$PreserveFQDN on +$RepeatedMsgReduction on + +$PrivDropToUser librenms +$PrivDropToGroup librenms + +# perform rsyslog magic instead of plain programname +# Juniper emits e.g. /kernel: as the tag, which makes programname +# write our own parser instead, that doesn't consider / as a terminator +$template librenms,%fromhost%||%syslogfacility-text%||%syslogpriority-text%||%syslogseverity-text%||%syslogtag%||%$year%-%$month%-%$day% %timereported:8:25%||%msg%||%syslogtag:R,ERE,1:([^:\[]+)(\[.*|:.*)?--end%\n +$ModLoad omprog +$ActionOMProgBinary /usr/local/sbin/librenms-syslog +*.* :omprog:;librenms diff --git a/modules/librenms/manifests/init.pp b/modules/librenms/manifests/init.pp index 07c3e11..24c3364 100644 --- a/modules/librenms/manifests/init.pp +++ b/modules/librenms/manifests/init.pp @@ -96,4 +96,9 @@ minute = '*/5', require = User['librenms'], } + +file { '/usr/local/sbin/librenms-syslog': +ensure = link, +target = ${install_dir}/syslog.php, +} } diff --git a/modules/librenms/manifests/syslog.pp b/modules/librenms/manifests/syslog.pp new file mode 100644 index 000..197a4da --- /dev/null +++ b/modules/librenms/manifests/syslog.pp @@ -0,0 +1,45 @@ +# == Class: librenms::syslog +# +# Sets up a separate rsyslog instance that receives messages in syslog (UDP +# 514) and forwards them to librenms' syslog script. +# +# Using a separate instance seems more complicated at first but provides +# certain important benefits: +# * The separate instance runs as the librenms user and hence is able to read +# the configuration file without giving access to the whole syslog group +# * There's no mixing of system syslog with the remote syslog and no messy +# filtering to avoid logging the local system's logs to LibreNMS +# * The received loglines are only