This is something that I just "discovered" as well.
Using a top-level filter is really a "post_filter" (it's renamed in later
versions of ES):
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-post-filter.html
So, this will execute the query first (a default "match_all: {}") and then
execute the filter on that result set. This is not very efficient for your
query, since I expect you expected having filter there to work like a
"pre-filter" and filter out results *before* executing the query.
To do that, you need to use a "filtered query":
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-filtered-query.html
In your case, the resulting query would look like:
curl -XGET 'http://localhost:9200/my-index/my-doc-type/_search' -d '{
"query": {
"filtered": {
"filter": {
"term": {"void": false}
}
}
},
"fields": [["user_id1", "user_name", "date", "status", "q1",
"q1_unique_code", "q2", "q3"]],
"size": 50000, "sort": ["date_value"]}'
On Fri, Nov 21, 2014 at 7:07 AM, Ajay Divakaran <[email protected]>
wrote:
> The term filter that is used:
>
> curl -XGET 'http://localhost:9200/my-index/my-doc-type/_search' -d '{
> "filter": {
> "term": {"void": false}
> },
> "fields": [["user_id1", "user_name", "date", "status", "q1",
> "q1_unique_code", "q2", "q3"]],
> "size": 50000, "sort": ["date_value"]}'
>
>
> - The 'void' field is a boolean field.
> - The index store size is 504mb.
> - The elastic search setup consists of only a single node and the
> index consists of only a single shard and 0 replicas. The version of
> elasticsearch is 0.90.7
> - The fields mentioned above is only the first 8 fields. The actual
> term filter that we execute has 350 fields mentioned.
>
> *We noticed the memory spiking by about 2-3gb though the store size is
> only 504mb.*
>
> *Running the query multiple times seems to continuously increase the
> memory.*
>
> Could someone explain why this memory spike occurs?
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/7c4ea660-9411-4d1d-a86c-84f1c43f4f7e%40googlegroups.com
> <https://groups.google.com/d/msgid/elasticsearch/7c4ea660-9411-4d1d-a86c-84f1c43f4f7e%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
--
Nick Canzoneri
Developer, Wildbit <http://wildbit.com/>
Beanstalk <http://beanstalkapp.com/>, Postmark <http://postmarkapp.com/>,
dploy.io
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKWm5yPQo5G0w2ShxtC4E9K90Bji1tQca3RqcG%2BvGUDR8aAqMQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
