Having a few issues with search performance. I was hoping there would be some way to get a detailed output on what phases of the search took all my time. I'm planning on passing the slowlog along to Kibana to set some idea of what is going on on a shard level, but thought I'd put this out there incase there was any way to get a breakdown along with the query response. i.e. I had to wait on shard X due to Y. Plus for highlighting any flaws in what I'm doing.
Setup: 6 data nodes, 256GB, 8xSSD, 60GB heap, G1GC Main search index: ~8TB + 1 replica. ~600mil docs (news articles) Other indices: 1TB + replicas, ~1bil docs Indexing: 150 docs/sec Querying: Full text, date histogram, faceting. Problems: - Some of our queries are quite large. Large near queries, up to 100 terms or'd together, neared with 5 terms, including wildcards. That type of thing. They take a long time, sometimes upwards of 3 mins. We want to be able to support this. 1 second responses maybe not, but I'd say 30 seconds at a max.. - These slow queries tend to prevent other queries from completing in sensible times, even for simple queries. - There's a large variance in query times. Sometimes 1 sec, sometimes much higher. I've tried to correlate to GCs and merges without success. Ideally I'd like to know what were queries waiting on, from which I can make adjustments. Obviously I could add more nodes to the cluster, but I'd like to know why I'm doing that rather than just taking a punt. Advise/abuse welcome. Harry -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1a668bd4-86a8-42d4-820c-a29d1be68762%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
