Also, I don't see a consistent response time from solr, I ran ab again and I get this:
ubuntu@ip-10-149-6-68:~$ ab -c 10 -n 500 " http://x.amazonaws.com:8983/solr/prodinfo/select?q=allText:huggies%20diapers%20size%201&rows=2000&wt=json " Benchmarking x.amazonaws.com (be patient) Completed 100 requests Completed 200 requests Completed 300 requests Completed 400 requests Completed 500 requests Finished 500 requests Server Software: Server Hostname: x.amazonaws.com Server Port: 8983 Document Path: /solr/prodinfo/select?q=allText:huggies%20diapers%20size%201&rows=2000&wt=json Document Length: 1538537 bytes Concurrency Level: 10 Time taken for tests: 10.858 seconds Complete requests: 500 Failed requests: 8 (Connect: 0, Receive: 0, Length: 8, Exceptions: 0) Write errors: 0 Total transferred: 769297992 bytes HTML transferred: 769268492 bytes Requests per second: 46.05 [#/sec] (mean) Time per request: 217.167 [ms] (mean) Time per request: 21.717 [ms] (mean, across all concurrent requests) Transfer rate: 69187.90 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.3 0 2 Processing: 110 215 72.0 190 497 Waiting: 91 180 70.5 152 473 Total: 112 216 72.0 191 497 Percentage of the requests served within a certain time (ms) 50% 191 66% 225 75% 252 80% 272 90% 319 95% 364 98% 420 99% 453 100% 497 (longest request) Sometimes it takes a lot of time, sometimes its pretty quick. Thanks, -Utkarsh On Fri, Jun 28, 2013 at 5:39 PM, Utkarsh Sengar <utkarsh2...@gmail.com>wrote: > Hello, > > I have a usecase where I need to retrive top 2000 documents matching a > query. > What are the parameters (in query, solrconfig, schema) I shoud look at to > improve this? > > I have 45M documents in 3node solrcloud 4.3.1 with 3 shards, with 30GB > RAM, 8vCPU and 7GB JVM heap size. > > I have documentCache: > <documentCache class="solr.LRUCache" size="1000000" > initialSize="1000000" autowarmCount="0"/> > > allText is a copyField. > > This is the result I get: > ubuntu@ip-10-149-6-68:~$ ab -c 10 -n 500 " > http://x.amazonaws.com:8983/solr/prodinfo/select?q=allText:huggies%20diapers%20size%201&rows=2000&wt=json > " > > Benchmarking x.amazonaws.com (be patient) > Completed 100 requests > Completed 200 requests > Completed 300 requests > Completed 400 requests > Completed 500 requests > Finished 500 requests > > > Server Software: > Server Hostname: x.amazonaws.com > Server Port: 8983 > > Document Path: > /solr/prodinfo/select?q=allText:huggies%20diapers%20size%201&rows=2000&wt=json > Document Length: 1538537 bytes > > Concurrency Level: 10 > Time taken for tests: 35.999 seconds > Complete requests: 500 > Failed requests: 21 > (Connect: 0, Receive: 0, Length: 21, Exceptions: 0) > Write errors: 0 > Non-2xx responses: 2 > Total transferred: 766221660 bytes > HTML transferred: 766191806 bytes > Requests per second: 13.89 [#/sec] (mean) > Time per request: 719.981 [ms] (mean) > Time per request: 71.998 [ms] (mean, across all concurrent requests) > Transfer rate: 20785.65 [Kbytes/sec] received > > Connection Times (ms) > min mean[+/-sd] median max > Connect: 0 0 0.6 0 8 > Processing: 9 717 2339.6 199 12611 > Waiting: 9 635 2233.6 164 12580 > Total: 9 718 2339.6 199 12611 > > Percentage of the requests served within a certain time (ms) > 50% 199 > 66% 236 > 75% 263 > 80% 281 > 90% 548 > 95% 838 > 98% 12475 > 99% 12545 > 100% 12611 (longest request) > > -- > Thanks, > -Utkarsh > -- Thanks, -Utkarsh
