If you are going to request 30,000 rows, you can give up on getting good performance. It is not going to happen.
Even without all the disk accesses, think about how much is sent over the network, then parsed by the client. The client cannot even start working with the data until it is all received and parsed. wunder On Jan 20, 2013, at 8:49 AM, Michael Ryan wrote: > (This is based on my knowledge of 3.6 - not sure if this has changed in 4.0) > > You are using rows=30000, which requires retrieving 30000 documents from > disk. In a non-distributed search, the QTime will not include the time it > takes to retrieve these documents, but in a distributed search, it will. For > a *:* query, the document retrieval will almost always be the slowest part of > the query. I'd suggest measuring how long it takes for the response to be > returned, or use rows=0. > > The timeAllowed feature is very misleading. It only applies to a small > portion of the query (which in my experience is usually not the part of the > query that is actually slow). Do not depend on timeAllowed doing anything > useful :) > > -Michael > > -----Original Message----- > From: Lyuba Romanchuk [mailto:lyuba.romanc...@gmail.com] > Sent: Sunday, January 20, 2013 6:36 AM > To: solr-user@lucene.apache.org > Subject: Solr 4.0 - timeAllowed in distributed search > > Hi, > > I try to use timeAllowed in query both in distributed search with one shard > and directly to the same shard. > I send the same query with timeAllowed=500 : > > - directly to the shard then QTime ~= 600 ms > - through distributes search to the same shard QTime ~= 7 sec. > > I have two questions: > > - It seems that timeAllowed parameter doesn't work for distributes > search, does it? > - What may be the reason that causes the query to the shard through > distributes search takes much more time than to the shard directly (the > same distribution remains without timeAllowed parameter in the query)? > > > Test results: > > Ask one shard through distributed search: > > > http://localhost:8983/solr/shard_2013-01-07/select?q=*:*&rows=30000&shards=127.0.0.1%3A8983%2Fsolr%2Fshard_2013-01-07&timeAllowed=500&partialResults=true&shards.info=true&debugQuery=true > <response> > <lst name="responseHeader"> > <bool name="partialResults">true</bool> > <int name="status">0</int> > <int name="QTime">7307</int> > <lst name="params"> > <str name="q">*:*</str> > <str name="shards">127.0.0.1:8983/solr/shard_2013-01-07</str> > <str name="partialResults">true</str> > <str name="debugQuery">true</str> > <str name="shards.info">true</str> > <str name="rows">30000</str> > <str name="timeAllowed">500</str></lst></lst> > <lst name="shards.info"> > <lst name="127.0.0.1:8983/solr/shard_2013-01-07"> > <long name="numFound">29574223</long> > <float name="maxScore">1.0</float> > <long name="time">646</long></lst></lst> > <result name="response" numFound="29574223" start="0" maxScore="1.0"> ... > 30,000 docs > ... > <lst name="debug"> > <str name="rawquerystring">*:*</str> > <str name="querystring">*:*</str> > <str name="parsedquery">MatchAllDocsQuery(*:*)</str> > <str name="parsedquery_toString">*:*</str> > <str name="QParser">LuceneQParser</str> > <lst name="timing"><double name="time">6141.0</double> <lst > name="prepare"><double name="time">0.0</double> <lst > name="org.apache.solr.handler.component.QueryComponent"><double > name="time">0.0</double></lst> > <lst name="org.apache.solr.handler.component.FacetComponent"><double > name="time">0.0</double></lst> > <lst name="org.apache.solr.handler.component.MoreLikeThisComponent"><double > name="time">0.0</double></lst> > <lst name="org.apache.solr.handler.component.HighlightComponent"><double > name="time">0.0</double></lst> > <lst name="org.apache.solr.handler.component.StatsComponent"><double > name="time">0.0</double></lst> > <lst name="org.apache.solr.handler.component.DebugComponent"><double > name="time">0.0</double></lst></lst> > <lst name="process"><double name="time">6141.0</double> <lst > name="org.apache.solr.handler.component.QueryComponent"><double > name="time">6022.0</double></lst> > <lst name="org.apache.solr.handler.component.FacetComponent"><double > name="time">0.0</double></lst> > <lst name="org.apache.solr.handler.component.MoreLikeThisComponent"><double > name="time">0.0</double></lst> > <lst name="org.apache.solr.handler.component.HighlightComponent"><double > name="time">0.0</double></lst> > <lst name="org.apache.solr.handler.component.StatsComponent"><double > name="time">0.0</double></lst> > <lst name="org.apache.solr.handler.component.DebugComponent"><double > name="time">119.0</double></lst></lst></lst>< > > Ask the same shard directly: > > http://localhost:8983/solr/shard_2013-01-07/select?q=*:*&rows=30000&timeAllowed=500&partialResults=true&shards.info=true&debugQuery=true > <lst name="responseHeader"> > <bool name="partialResults">true</bool> > <int name="status">0</int> > <int name="QTime">617</int> > <lst name="params"> > <str name="q">*:*</str> > <str name="partialResults">true</str> > <str name="debugQuery">true</str> > <str name="shards.info">true</str> > <str name="rows">30000</str> > <str name="timeAllowed">500</str></lst></lst> > <result name="response" numFound="28687243" start="0"> ... > 30,000 docs > <lst name="debug"><str name="rawquerystring">*:*</str><str > name="querystring">*:*</str><str > name="parsedquery">MatchAllDocsQuery(*:*)</str><str > name="parsedquery_toString">*:*</str> > <str name="QParser">LuceneQParser</str> > <lst name="timing"><double name="time">617.0</double> <lst > name="prepare"><double name="time">0.0</double> <lst > name="org.apache.solr.handler.component.QueryComponent"><double > name="time">0.0</double></lst> > <lst name="org.apache.solr.handler.component.FacetComponent"><double > name="time">0.0</double></lst> > <lst name="org.apache.solr.handler.component.MoreLikeThisComponent"><double > name="time">0.0</double></lst> > <lst name="org.apache.solr.handler.component.HighlightComponent"><double > name="time">0.0</double></lst> > <lst name="org.apache.solr.handler.component.StatsComponent"><double > name="time">0.0</double></lst> > <lst name="org.apache.solr.handler.component.DebugComponent"><double > name="time">0.0</double></lst></lst> > <lst name="process"><double name="time">617.0</double> <lst > name="org.apache.solr.handler.component.QueryComponent"><double > name="time">516.0</double></lst> > <lst name="org.apache.solr.handler.component.FacetComponent"><double > name="time">0.0</double></lst> > <lst name="org.apache.solr.handler.component.MoreLikeThisComponent"><double > name="time">0.0</double></lst> > <lst name="org.apache.solr.handler.component.HighlightComponent"><double > name="time">0.0</double></lst> > <lst name="org.apache.solr.handler.component.StatsComponent"><double > name="time">0.0</double></lst> > <lst name="org.apache.solr.handler.component.DebugComponent"><double > name="time">101.0</double></lst></lst></lst></lst> > > Thank you. > Best regards, > Lyuba -- Walter Underwood wun...@wunderwood.org
