I'm new to solr and I'm load testing our setup to see what we can handle. I'm using solrmeter and my problem is a bit odd: * When I set solrmeter to run 4000 queries/min, it will handle a few hundred queries and then tomcat will stop responding completely to requests (even though according to lsof -i it is still listening and the java process is still running). * When I set solrmeter to run 1000 queries/min it runs fine. I can stop solrmeter after a couple of minutes at that pace and then run at 4000/min without issue.
It's as if it needs a ramp up time? Also, I noticed (regardless of ramp up) that my setup cannot handle 8000/min. The reaction at 8k/min is the same as if I were to run 4k/min without the ramp up. Of note, only the shard that solrmeter is pointed to stops responding. The other shard hums along without incident. Setup (everything in AWS): - 2x m1.large (7.5Gb RAM) running tomcat7 + solr 4.2.0 (open-jdk-7-headless) : Ubuntu 12.04 - 1x m1.micro running zookeeper 3.4.5 : Ubuntu 12.04 I have ~30k documents in each node (~300Mb on each node) The vast majority of my solr/tomcat7 config is default from ubuntu's packages/solr's example dir. Here's the configs and the end of the catalina.out file:https://gist.github.com/anonymous/ef8fa79ecc1673d11bc0 My main question is two fold: 1. Is this normal behavior for tomcat (to just stop responding completely) when it gets overwhelmed? And the only option is to restart it? I guess I dont know what it looks like when tomcat/solr cant keep up. 2. Why does it handle better when I give it a lower number of queries and then ramp it up? It concerns me that if I have to restart a server in the cluster and it gets thrown into the pool of machines that things will blow up. As an aside, does this seem like a normal amount of queries (~4k/min) that this kind of environment should be able to handle?
