Hi Ajay, As Nick pointed out, this query is going to match documents on a doc-by-doc basis, which is going to be very slow.
However, each iteration is not supposed to increase memory usage. Memory usage might jump on the first request because elasticsearch will need to load the `date_value` field into field data and potentially your term filter into the filter cache, but that should be it, and subsequent executions of this request should not add 2GB of garbage. Something that is uncommonly high in your query is the `size` parameter. Do you have an idea of how large your documents are? It could be that part of the reason why so much garbage is generated is due to the building and then serialization of the search response. On Fri, Nov 21, 2014 at 1:07 PM, Ajay Divakaran <[email protected]> wrote: > The term filter that is used: > > curl -XGET 'http://localhost:9200/my-index/my-doc-type/_search' -d '{ > "filter": { > "term": {"void": false} > }, > "fields": [["user_id1", "user_name", "date", "status", "q1", > "q1_unique_code", "q2", "q3"]], > "size": 50000, "sort": ["date_value"]}' > > > - The 'void' field is a boolean field. > - The index store size is 504mb. > - The elastic search setup consists of only a single node and the > index consists of only a single shard and 0 replicas. The version of > elasticsearch is 0.90.7 > - The fields mentioned above is only the first 8 fields. The actual > term filter that we execute has 350 fields mentioned. > > *We noticed the memory spiking by about 2-3gb though the store size is > only 504mb.* > > *Running the query multiple times seems to continuously increase the > memory.* > > Could someone explain why this memory spike occurs? > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/7c4ea660-9411-4d1d-a86c-84f1c43f4f7e%40googlegroups.com > <https://groups.google.com/d/msgid/elasticsearch/7c4ea660-9411-4d1d-a86c-84f1c43f4f7e%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- Adrien Grand -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j5pdoVmgSp-27pWQvzJ-aHWqbKDAubmuMN9RUVWCvndDw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
