I think we could try that, but most likely it turns out that at some
point we are receiving 300 requests per second, and are able to
reasonably handle 150 per second, which means everything else is going
to be kept in the growing queue and increase response times even further..
Also, if one node has 12 cores that would mean it can process 12
concurrent searches? And since every request is sent to all shards to
check if there are results, does this also mean the whole cluster can
handle 12 concurrent requests on average?
On 27.10.18 09:00, Toke Eskildsen wrote:
Sofiya Strochyk <s...@interlogic.com.ua> wrote:
Target query rate is up to 500 qps, maybe 300, and we need
to keep response time at <200ms. But at the moment we only
see very good search performance with up to 100 requests
per second. Whenever it grows to about 200, average response
time abruptly increases to 0.5-1 second.
Keep in mind that upping the number of concurrent searches in Solr does not
raise throughput, if the system is already saturated. On the contrary, this
will lower throughput due to thread- and memory-congestion.
As your machines has 12 cores (including HyperThreading) and IO does not seem
to be an issue, 500 or even just 200 concurrent searches seems likely to result
in lower throughput than (really guessing here) 100 concurrent searches. As
Walther point out, the end result is collapse, but slowdown happens before that.
Consider putting a proxy in front with a max amount of concurrent connections
and a sensible queue. Preferably after a bit of testing to locale where the
highest throughput is. It won't make you hit your overall goal, but it can move
you closer to it.
- Toke Eskildsen
--
Email Signature
*Sofiia Strochyk
*
s...@interlogic.com.ua <mailto:s...@interlogic.com.ua>
InterLogic
www.interlogic.com.ua <https://www.interlogic.com.ua>
Facebook icon <https://www.facebook.com/InterLogicOfficial> LinkedIn
icon <https://www.linkedin.com/company/interlogic>
