Gehel has submitted this change and it was merged.
Change subject: logstash: Remove all _* fields from gelf records
......................................................................
logstash: Remove all _* fields from gelf records
Fields named _id, _index and _type were retained when GELF filtering was
initially setup in dfb0a19. These field names are significant in
Elasticsearch, but they are added by the Elasticsearch server itself as
metadata for each log event passed in by Logstash. They should not be
passed in via the application. Doing so results in mapping conflicts
that discard the entire event such as "MapperParsingException[failed to
parse]; nested: IllegalStateException[Mixing up field types: class
org.elasticsearch.index.mapper.core.StringFieldMapper$StringFieldType !=
class org.elasticsearch.index.mapper.internal.IdFieldMapper$IdFieldType
on field _id];".
Change-Id: I6306f76052c163e1edbf044343d11514b5b20803
---
M files/logstash/filter-gelf.conf
1 file changed, 3 insertions(+), 3 deletions(-)
Approvals:
Gehel: Looks good to me, approved
EBernhardson: Looks good to me, but someone else must approve
jenkins-bot: Verified
diff --git a/files/logstash/filter-gelf.conf b/files/logstash/filter-gelf.conf
index ed39bb4..1ce9164 100644
--- a/files/logstash/filter-gelf.conf
+++ b/files/logstash/filter-gelf.conf
@@ -84,10 +84,10 @@
remove_field => [ "full_message" ]
}
- # Get rid of a whole bunch of useless _* fields that are duplicated.
- # We must keep _id, _index, and _type
+ # Get rid of a whole bunch of useless _* fields that are duplicated or
+ # collide with internal Elasticsearch meta data.
prune {
- blacklist_names => [ "^_(?!id$|index$|type$).*" ]
+ blacklist_names => [ "^_.*" ]
}
} # end [type] == "gelf"
--
To view, visit https://gerrit.wikimedia.org/r/298382
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I6306f76052c163e1edbf044343d11514b5b20803
Gerrit-PatchSet: 3
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: BryanDavis <[email protected]>
Gerrit-Reviewer: BryanDavis <[email protected]>
Gerrit-Reviewer: EBernhardson <[email protected]>
Gerrit-Reviewer: Gehel <[email protected]>
Gerrit-Reviewer: jenkins-bot <>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
