When fielddata is loaded, is it only that of which the aggregation job needs (items.question_label.raw in this case) or does it load the full _source of every match and extract the field?
On Monday, April 28, 2014 9:04:09 PM UTC-4, Olivier B wrote: > > Thanks you. > OK, that's what I was fearing: the cache is loaded regardless of the > filter condition. Which is a shame, even if we filter a lot, targeting only > one document, we still need to fill up the cache! > I will try to have a lot of RAM and see if I'm reaching a stable memory > occupation and let the cache living like that. > Alternative solution is to have many indexes, each index will act as a > pre-filter and contains way less data. > Do you know if the fielddata cache is loading all docs, or only the > relevant shard? Would it help to have smaller shards? > > On Monday, April 28, 2014 11:55:22 PM UTC+10, Binh Ly wrote: >> >> You are correct. Unfortunately the fielddata is loaded for all docs >> regardless of filter condition. You can: >> >> 1) Add more RAM >> >> 2) Add more nodes (and shard your index out so that RAM usage will >> distributed across multiple nodes) >> >> 3) Use disk-based fielddata (fielddata will not be loaded into memory) >> for the field/s you are aggregating on. This will run slower and you have >> to reindex your data. >> >> http://www.elasticsearch.org/blog/disk-based-field-data-a-k-a-doc-values/ >> > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/352608c0-ffbe-4fbd-ab5e-9c5809137bb0%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
