Gage has submitted this change and it was merged.
Change subject: logstash: Rules for processing MW input via Redis
......................................................................
logstash: Rules for processing MW input via Redis
Add a filter to tag MediaWiki log events received from Redis for storage
in Elasticsearch.
Bug: T845
Change-Id: I140e3127a10c6e7561951ad68b6fb2bce2414e0e
---
A files/logstash/filter-mw-via-redis.conf
M manifests/role/logstash.pp
M modules/logstash/templates/input/redis.erb
3 files changed, 19 insertions(+), 0 deletions(-)
Approvals:
Gage: Looks good to me, approved
jenkins-bot: Verified
diff --git a/files/logstash/filter-mw-via-redis.conf
b/files/logstash/filter-mw-via-redis.conf
new file mode 100644
index 0000000..c9acec5
--- /dev/null
+++ b/files/logstash/filter-mw-via-redis.conf
@@ -0,0 +1,13 @@
+# vim:set sw=2 ts=2 sts=2 et
+# Process MediaWiki log output sent via the Redis "logstash" queue
+filter {
+
+ if [type] == "mediawiki" and "redis" in [tags] {
+ # Tag for storage in elasticsearch
+ mutate {
+ add_tag => [ "es" ]
+ }
+
+ } # end [type] == "mediawiki"
+
+}
diff --git a/manifests/role/logstash.pp b/manifests/role/logstash.pp
index ddd377a..01a38f7 100644
--- a/manifests/role/logstash.pp
+++ b/manifests/role/logstash.pp
@@ -89,6 +89,11 @@
priority => 50,
}
+ logstash::conf { 'filter_mw_via_redis':
+ source => 'puppet:///files/logstash/filter-mw-via-redis.conf',
+ priority => 50,
+ }
+
logstash::conf { 'filter_gelf':
source => 'puppet:///files/logstash/filter-gelf.conf',
priority => 50,
diff --git a/modules/logstash/templates/input/redis.erb
b/modules/logstash/templates/input/redis.erb
index cb33907..ed5629a 100644
--- a/modules/logstash/templates/input/redis.erb
+++ b/modules/logstash/templates/input/redis.erb
@@ -7,5 +7,6 @@
<%- if @password -%>
password => "<%= @password %>"
<%- end -%>
+ tags => [ "redis" ]
}
}
--
To view, visit https://gerrit.wikimedia.org/r/175896
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I140e3127a10c6e7561951ad68b6fb2bce2414e0e
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: BryanDavis <[email protected]>
Gerrit-Reviewer: BryanDavis <[email protected]>
Gerrit-Reviewer: Gage <[email protected]>
Gerrit-Reviewer: jenkins-bot <>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
