Hi all,
I'm currently in the process of doing some performance testing in preparations
for upgrading from Solr 3.6.1 to Solr 4.0. (We're badly in need of NRT
functionality)
Our existing deployment is not a typical deployment for Solr, as we use it to
search and facet on financial data such as accounts, positions and transactions
records. To make matters worse, each request could potentially return upwards
of 50,000 or more records from the index. As I said, it's not an ideal use case
for Solr but this is the system that is in place and it really can't be changed
at this point. With this defined use case, our current 3.6.1 deployment is able
to scale to about 1500 queries per minute, with an average response time in the
low 100-200ms. Note that this time includes the query time and the transport
time (time to stream all the documents to the calling services). At the 50,000
document mark, we're getting about 1.6-2 sec. response time. The client is
willing to live with this as these type of requests are not very frequent.
Our hardware configuration on the 3.6.1 environment is as follows:
* 1 Master Server for indexing with 2 CPU (each 6 cores, 2.67GHz) 4GB
of RAM and 150GB HDD
* 2 Slaves Servers for query only each with 2 of CPUs (each 6 cores,
2.67GHz) with 12GB of RAM each and same HDD space. (mechanical drive)
Each of the servers are virtual servers in a VMWare environment.
Now with the roughly the "same" schema and solrconfig configuration, the
performance on Solr 4.0 is quite bad. Running just 500 queries per minute our
query performance degrades to almost 2 minute response times in some cases. The
average is about 40-50 sec. response time. Note that the index at the moment is
only a fraction of the size of the existing environment (about 1/8th the size).
The hardware setup for the SolrCloud deployment is as follows:
* 4 Solr server instances each with 4 CPUs (each 6 cores, 2.67GHz), 8GB
of RAM and 150GB HDD
* 3 ZooKeeper server instances. We are using each Solr server instance
to run 1 ZK instance, with the 4th server not running a ZK server.
We haven't observed any issues with memory utilization. Additionally the
virtual servers are co-located. We're wondering if upgrading to Solid State
Drives would improve performance significantly?
Are there any other pointers or configuration changes that we can make to help
bring down our query times? Any tips will be greatly appreciated.
Thanks all!
