Thanks you. OK, that's what I was fearing: the cache is loaded regardless of the filter condition. Which is a shame, even if we filter a lot, targeting only one document, we still need to fill up the cache! I will try to have a lot of RAM and see if I'm reaching a stable memory occupation and let the cache living like that. Alternative solution is to have many indexes, each index will act as a pre-filter and contains way less data. Do you know if the fielddata cache is loading all docs, or only the relevant shard? Would it help to have smaller shards?
On Monday, April 28, 2014 11:55:22 PM UTC+10, Binh Ly wrote: > > You are correct. Unfortunately the fielddata is loaded for all docs > regardless of filter condition. You can: > > 1) Add more RAM > > 2) Add more nodes (and shard your index out so that RAM usage will > distributed across multiple nodes) > > 3) Use disk-based fielddata (fielddata will not be loaded into memory) for > the field/s you are aggregating on. This will run slower and you have to > reindex your data. > > http://www.elasticsearch.org/blog/disk-based-field-data-a-k-a-doc-values/ > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6a46345d-da2e-403c-8c9f-d47de4b70bac%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
