It might be useful to fiddle with the merge configuration <http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-merge.html> to try to end up with fewer segments. That'll reduce the IO cost of the underlying Lucene operations that filter your query before the aggregations. One win is to make sure you aren't oversubscribing. So if you are going for maximum speed have one shard per server. Maybe one less. If you are going for maximum throughput (like, total queries) then have fewer total copies of the data then you do servers. So if you have 5 shards with 2 replicas, you'd need at least 15 servers.
Cutting down the number of really helped my throughput but it might have been because my workload is different. Nik On Thu, Jul 17, 2014 at 11:56 AM, jnortey <[email protected]> wrote: > At the moment, we're able to bulk index data at a rate faster than we > actually need. Indexing is not as important to use as being able to quickly > search for data. Once we start reaching ~30 million documents indexed, we > start to see performance decreasing in ours search queries. What are the > best techniques for sacrificing indexing time in order to improve search > performance? > > > A bit more info: > > - We have the resources to improve our hardware (memory, CPU, etc) but > we'd like to maximize the improvements that can be made programmatically or > using properties before going for hardware increases. > > - Our searches make very heavy uses of faceting and aggregations. > > - When we run the optimize query, we see *significant* improvements in > our search times (between 50% and 80% improvements), but as documented, > this is usually a pretty expensive operation. Is there a way to sacrifice > indexing time in order to have Elasticsearch index the data more > efficiently? (I guess sort of mimicking the optimization behavior at index > time) > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/0e134001-9a55-40c5-a8fc-4c1485a3e6fc%40googlegroups.com > <https://groups.google.com/d/msgid/elasticsearch/0e134001-9a55-40c5-a8fc-4c1485a3e6fc%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1WEWDomC20wvP0nja_dtEwDtmNFTfa5fp0AOeirShowA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
