Seeing memory continuously increasing over a couple of requests is not necessarily a bad sign. If you give X gigabytes of memory to a JVM, it won't hesitate to use them if it can help decrease the frequency at which it has to run costly garbage collections. What is more important to watch is how memory usage behaves over a long period, eg. does the frequency at which GCs run keep on increasing (which could indicate that the server is encountering memory pressure or that there is something that leaks memory somewhere).
On Fri, Nov 21, 2014 at 6:50 PM, Ajay Divakaran <[email protected]> wrote: > Thanks Adrien for your reply. > > I upgraded to ES 0.90.13. What I'm now noticing is that the memory seems > to continuously increase when running the query again and again. > I also noticed that after upgrading it started using OpenJDK 1.6_0.33. So > I switched back to using Oracle JDK 1.7.71 however the issue seems to > persist. > > > On Friday, November 21, 2014 10:57:09 PM UTC+5:30, Adrien Grand wrote: >> >> One difference is that the store is compressed while the structure of the >> response that is built into memory is not (and potentially very wasteful). >> >> The doc-by-doc comparison is done by the post-filter: for every document >> that matches the query, the filter is evaluated in order to know whether it >> matches or not. On the other hand, when a filter is in the query (either >> under a constant_score or a filtered_query), it can efficiently jump to the >> next matches using the inverted index. >> >> If you want to export your index, a more efficient way would be to use >> the `scan` search type: http://www.elasticsearch.org/ >> guide/en/elasticsearch/guide/current/scan-scroll.html. It basically >> opens a cursor that you can iterate on, as opposed as trying to get >> everything at once. >> >> On Fri, Nov 21, 2014 at 4:43 PM, Ajay Divakaran <[email protected]> >> wrote: >> >>> Each document has around 350 text fields. >>> >>> But im still not able to relate the store size to the memory spike. >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elasticsearch" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit https://groups.google.com/d/ >>> msgid/elasticsearch/c8014753-b0e2-481c-85dc-8a9eceae7d32% >>> 40googlegroups.com. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> >> >> -- >> Adrien Grand >> > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/77ab05ed-39cd-4313-8d1a-9950995e8b71%40googlegroups.com > <https://groups.google.com/d/msgid/elasticsearch/77ab05ed-39cd-4313-8d1a-9950995e8b71%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- Adrien Grand -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j4zPn5ECjtcVCXjjmfNNbYyJ47HRsVdrvA%3DYHaoP-ZfbQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
