- Fielddata (sorting, facets, some scripting), filter cache (filters), and id_cache (parent-child) are probably the ones that would affect memory usage significantly. - Fielddata can be loaded in different ways depending on data type and other config (details here: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/fielddata-formats.html#_fielddata_loading) - OS memory would generally be file system caches for the underlying Lucene indexes
About your other question, the default query execution is called query_then_fetch (http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-search-type.html#query-then-fetch). The basic idea is the search (query) is scattered in parallel to all shards (single replica set) of an index being searched. Then a little bit of results are gathered back, reduced, and then sorted. Then whatever is the final reduced list, ES goes back to the final shards and pulls out (fetch) additional data required to return final hits to the caller. On Wednesday, March 26, 2014 10:41:44 AM UTC-4, Uli Bethke wrote: > > I am struggling a bit to understand which ES operations use up which part > of memory > > - From reading the documentation I know that faceting and sorting take up > heap memory and also that some filters are cached in JVM heap. However, > what other operations take up JVM heap memory? > - When faceting on a particular field are all of the values of that field > loaded into JVM heap or only the distinct values of the field. I would > believe it is the latter as fields of high cardinality require more JVM > heap. > - What operations use OS memory. I believe documents retrieved and ES > index are kept in OS memory. Can anyone confirm this? What other operations > go to OS memory? > > > Finally, I would also be interested if there are any posts or tutorials > that desribce the complete lifecycle of a query from submission of query to > return of resultset. > > Thanks > Uli > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/dbe09764-67e8-49cd-98e7-9c9b5e4a801a%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
