Hello, The search threadpool size (that is, how many requests can be actually worked on at once) defaults to 3 times the number of processors. This might be reduced in future, though, see: https://github.com/elasticsearch/elasticsearch/pull/9165
The queue size (how many requests ES can accept before starting to reject them) defaults to 1000. >From what I understand, this is per thread, so the answer to your question depends on how many processors you have and how many shards get hit by each search. For example, if a search runs on 3 indices, each with 2 shards (number of replicas is irrelevant, because the search will only hit one complete set of data) you'll get 6 requests in the threadpool per search. If you have two servers with 8 cores each, you 8*3*2=48 threads available. So the cluster can work on 8 requests at once. On top of that it can still queue round-down(2 nodes * 1000 queue size/6 requests per search)=333 searches until it starts rejecting them. Regarding your scaling question, I can't give you a direct answer, unfortunately, because it depends on a whole lot of variables, mainly how your data, queries and hardware look like and what can be changed. The fact that your threadpool queue got full is just a symptom, it's not clear to me what happens in there. I usually see this when there are lots of indices and/or those indices have lots of shards. So a single request takes a lot of requests in the threadpool, filling it up, even if the ES cluster can keep up with the load. If that's your case increase the threadpool queue size and make sure you don't have too many shards per index. If your cluster can't keep up with the load (a monitoring tool like SPM <http://sematext.com/spm/> should show you that), then the first step is to see where is the bottleneck. Again, monitoring can give some insight: are queries too expensive, can they be optimized? do you have too many cache evictions? is the heap size too large or too small? is memory, I/O or CPU the bottleneck? Things like that. It could also be that you need more/different hardware. Finally, you can make scaling ES someone else's problem by using a hosted service like Logsene <http://sematext.com/logsene/index.html>. Especially if you're using ES for log- or metric-like data, you'll get lots of features out of the box, and we expose most of the ES API to plug in your custom stuff. Best regards, Radu -- Performance Monitoring * Log Analytics * Search Analytics Solr & Elasticsearch Support * http://sematext.com/ On Thu, Jan 8, 2015 at 1:32 PM, vipins <[email protected]> wrote: > What is the maximum limit on the concurrent search requests with default > Elastic search server settings. > > I am able to perform only 5 parallel search requests in my application with > default settings. > > how can we improve the scalability of ES server search requests apart from > increasing number of node,shards and queue size in search thread pool. > > thanks. > > > > -- > View this message in context: > http://elasticsearch-users.115913.n3.nabble.com/concurrent-search-request-to-elasticsearch-tp4068702.html > Sent from the ElasticSearch Users mailing list archive at Nabble.com. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/1420716748150-4068702.post%40n3.nabble.com > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAHXA0_2ZoQ6bewJ_aSxFMxAO0_5Mdsu%3D9WA4m_be7AfSb%3D2%2BTg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
