BryanDavis has uploaded a new change for review.
https://gerrit.wikimedia.org/r/176693
Change subject: logstash: Forward syslog events for apache2 + hhvm
......................................................................
logstash: Forward syslog events for apache2 + hhvm
Forward syslog events from apache2 and hhvm processes to a logstash
server if configured to do so via hiera. Forwards the same events that
would normally be sent to a udp2log relay directly to logstash.
Using the syslog input to logstash and rsylog as the transfer agent from
the individual MediaWiki hosts moves us closer to being able to
discontinue the use of log2udp forwarding to get events into logstash.
Bug: T76119
Change-Id: I5e8ea2b9c917ced306631c8092ce3149920a28f1
---
M hieradata/eqiad.yaml
M modules/mediawiki/manifests/init.pp
M modules/mediawiki/templates/rsyslog.conf.erb
3 files changed, 20 insertions(+), 1 deletion(-)
git pull ssh://gerrit.wikimedia.org:29418/operations/puppet
refs/changes/93/176693/1
diff --git a/hieradata/eqiad.yaml b/hieradata/eqiad.yaml
index ab0dd5a..f7ab6e1 100644
--- a/hieradata/eqiad.yaml
+++ b/hieradata/eqiad.yaml
@@ -16,3 +16,6 @@
- '10.64.0.194:11211:1'
- '10.64.0.195:11211:1'
+# TODO: setup something to loadbalance the logstash hosts rather than relying
+# on a single node in the cluster.
+"mediawiki::logstash_syslog": "logstash1001.eqiad.wmnet:10514"
diff --git a/modules/mediawiki/manifests/init.pp
b/modules/mediawiki/manifests/init.pp
index b3fa32c..39be569 100644
--- a/modules/mediawiki/manifests/init.pp
+++ b/modules/mediawiki/manifests/init.pp
@@ -12,8 +12,12 @@
# [*log_aggregator*]
# Udp2log aggregation server to send logs to. Default 'udplog:8420'.
#
+# [*logstash_syslog*]
+# Logstash aggregation server to forward syslog events to. Default undef.
+#
class mediawiki (
- $log_aggregator = 'udplog:8420',
+ $log_aggregator = 'udplog:8420',
+ $logstash_syslog = undef,
) {
include ::mediawiki::cgroup
include ::mediawiki::packages
diff --git a/modules/mediawiki/templates/rsyslog.conf.erb
b/modules/mediawiki/templates/rsyslog.conf.erb
index e7b4b0b..60ac501 100644
--- a/modules/mediawiki/templates/rsyslog.conf.erb
+++ b/modules/mediawiki/templates/rsyslog.conf.erb
@@ -13,16 +13,28 @@
:programname, isequal, "apache2" :omfile:$apache2
& @<%= scope['::mediawiki::log_aggregator'] %>;MediaWiki
+<%- if scope['::mediawiki::logstash_syslog'] %>
+& @%= scope['::mediawiki::logstash_syslog'] %>
+<% end -%>
& ~
# Forward HHVM logs and stack traces to log aggregator.
:programname, isequal, "hhvm" @<%= scope['::mediawiki::log_aggregator']
%>;MediaWiki
+<%- if scope['::mediawiki::logstash_syslog'] %>
+& @%= scope['::mediawiki::logstash_syslog'] %>
+<% end -%>
& ~
:syslogtag, isequal, "hhvm-fatal:" @<%= scope['::mediawiki::log_aggregator']
%>;MediaWiki
+<%- if scope['::mediawiki::logstash_syslog'] %>
+& @%= scope['::mediawiki::logstash_syslog'] %>
+<% end -%>
& ~
# Forward messages logged by the kernel and containing the string
# "hhvm" (such as warnings that the process was killed or respawned)
# to the log aggregator.
if $msg contains "hhvm" and $programname == "kernel" then @<%=
scope['::mediawiki::log_aggregator'] %>;MediaWiki
+<%- if scope['::mediawiki::logstash_syslog'] %>
+if $msg contains "hhvm" and $programname == "kernel" then @%=
scope['::mediawiki::logstash_syslog'] %>
+<% end -%>
--
To view, visit https://gerrit.wikimedia.org/r/176693
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I5e8ea2b9c917ced306631c8092ce3149920a28f1
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: BryanDavis <[email protected]>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
