We've configured logstash in combination with elasticsearch and Kibana to
centralize our server logs.
In Kibana I want to setup a table which groups all error messages so we can
create a top 10 of the most occuring errors.
We tried to setup a terms table grouped on a specific field (which contains
the error message).
However there is a record "missing fields" in the table with a count of
16997.
The configuration of the terms table results in the following elasticsearch
query (posting a part of the filter):
"facets": {
"terms": {
"terms": {
"field": "information.information.raw",
"size": 100000000,
"order": "count",
"exclude": []
}
[...]
In the result a missing property indicates quite a lot of missing results
as we've noticed in the terms table itself:
"facets": {
"terms": {
"_type": "terms",
"missing": 16997,
"total": 584,
"other": 0,
[...]
I've noticed that in the mapping defined by logstash the following is added:
"information.raw": {
"type": "string",
"index": "not_analyzed",
"ignore_above": 256
}
I assume that ignore_above means that any message longer then 256
characters will not be taken into account when doing queries?
I tried to find some documentation about it but the only thing I can find
is at the Core Types documentation
<http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-core-types.html>
of elasticsearch and I'm not entirely sure if I understand it correctly.
If I remove this property in the mapping, what could be the impact of this?
Slower queries because it needs to take larger strings into account? Is if
safe to remove this property?
Thanks in advance!
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/3a5f188f-e77e-458a-ab34-eaf913f8b291%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
