Filippo Giunchedi has submitted this change and it was merged. (
https://gerrit.wikimedia.org/r/313035 )
Change subject: Use logstash's prune filter for api-feature-usage-sanitized
......................................................................
Use logstash's prune filter for api-feature-usage-sanitized
We didn't do this originally because logstash 1.4 has a bug in matching
cloned events. That's fixed in 1.5, so we can use prune{} to whitelist
only the fields we actually need.
Change-Id: Ifa6471958f43e20800e0a439f78a26fc51d514e2
---
M modules/role/files/logstash/filter-apifeatureusage.conf
1 file changed, 5 insertions(+), 11 deletions(-)
Approvals:
BryanDavis: Looks good to me, but someone else must approve
Filippo Giunchedi: Verified; Looks good to me, approved
DCausse: Looks good to me, but someone else must approve
diff --git a/modules/role/files/logstash/filter-apifeatureusage.conf
b/modules/role/files/logstash/filter-apifeatureusage.conf
index ca4245f..cd577d9 100644
--- a/modules/role/files/logstash/filter-apifeatureusage.conf
+++ b/modules/role/files/logstash/filter-apifeatureusage.conf
@@ -3,19 +3,13 @@
filter {
if [type] == "mediawiki" and [channel] == "api-feature-usage" and
!("_grokparsefailure" in [tags]) {
# Make a sanitized copy for Extension:ApiFeatureUsage
- # @todo Logstash 1.4.x has a bug where later filter-code will not see
- # the changes to the copied event made by clone{}, but it will be
- # reflected in the output. If we upgrade to 1.5 and it's really fixed
- # there, we can use prune{} to whitelist fields instead.
clone {
clones => [ "api-feature-usage-sanitized" ]
- remove_field => [
- # Important to remove
- "username", "ip", "referer",
- # Not so important, but unneeded
- "host", "normalized_message", "sequence_id", "wikidb", "udp_sender"
- ]
- remove_tag => [ 'es', 'udp2log','logdate','normalized_message_untrimmed'
]
+ }
+ if [type] == "api-feature-usage-sanitized" {
+ prune {
+ whitelist_names => [ 'type', '@timestamp', '@version', 'feature',
'agent' ]
+ }
}
} # end [channel] == "api-feature-usage"
}
--
To view, visit https://gerrit.wikimedia.org/r/313035
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Ifa6471958f43e20800e0a439f78a26fc51d514e2
Gerrit-PatchSet: 5
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Anomie <bjor...@wikimedia.org>
Gerrit-Reviewer: Anomie <bjor...@wikimedia.org>
Gerrit-Reviewer: BryanDavis <bda...@wikimedia.org>
Gerrit-Reviewer: DCausse <dcau...@wikimedia.org>
Gerrit-Reviewer: Filippo Giunchedi <fgiunch...@wikimedia.org>
Gerrit-Reviewer: Gehel <gleder...@wikimedia.org>
Gerrit-Reviewer: jenkins-bot <>
_______________________________________________
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
