You should be using docvalues on your id, but note that switching this would require a reindex.
On Wed, Jan 13, 2021 at 6:04 AM Ajay Sharma <aja...@indiamart.com.invalid> wrote: > Hi All, > > I have used cursors to search and export documents in solr according to > > https://lucene.apache.org/solr/guide/6_6/pagination-of-results.html#fetching-a-large-number-of-sorted-results-cursors > > Solr version: 6.5.0 > No of Documents: 10 crore > > Before implementing cursor, I was using the start and rows parameter to > fetch records > Service response time used to be 2 sec > > *Before implementing Cursor Solr URL:* > http://localhost:8080/solr/search/select?q=bird > toy&qt=mapping&ps=3&rows=25&mm=100 > > Request handler Looks like this: fl contains approx 20 fields > <requestHandler name="mapping" class="solr.SearchHandler"> > <lst name="invariants"> > <str name="defType">edismax</str> > <str name="indent">on</str> > <float name="tie">0.01</float> > </lst> > <lst name="appends"> > <str name="fl">id,refid,title,smalldesc:""</str> > </lst> > <lst name="defaults"> > <str name="echoParams">none</str> > <str name="wt">json</str> > <int name="rows">25</int> > <str name="timeAllowed">15000</str> > <str name="qf">smalldesc</str> > <str name="qf">title_text</str> > <str name="qf">titlews^3</str> > <str name="qf">sdescnisq</str> > <str name="qs">1</str> > <!-- retrive following fields --> > <str name="mm">2<-1 4<70%</str> > </lst> > </requestHandler> > > Sharing Response with EchoParams=all > Qtime is 6 > responseHeader: { > status: 0, > QTime: 6, > params: { > ps: "3", > echoParams: "all", > indent: "on", > fl: "id,refid,title,smalldesc:"", > tie: "0.01", > defType: "edismax", > qf: "customphonetic", > wt: "json", > qs: "1", > qt: "mapping", > rows: "25", > q: "bird toy", > timeAllowed: "15000" > } > }, > response: { > numFound: 17, > start: 0, > maxScore: 26.616478, > docs: [ > { > id: "22347708097", > refid: "152585558", > title: "Round BIRD COLOURFUL SWINGING CIRCULAR SITTING TOY", > smalldesc: "", > score: 26.616478 > } > ] > } > > I am facing a performance issue now after implementing the cursor. Service > response time is increased 3 to 4 times .i.e. 8 sec in some cases > > *After implementing Cursor query is-* > localhost:8080/solr/search/select?q=bird > toy&qt=cursor&ps=3&rows=1000&mm=100&sort=score desc,id asc&cursorMark=* > > Just added &sort=score desc,id asc&cursorMark=* to the before query and > rows to be fetched is 1000 now and fl contains just a single field > > Request handler remains same as before just changed the name and made fl > change and added df in defaults > > <requestHandler name="cursor" class="solr.SearchHandler"> > <lst name="invariants"> > <str name="defType">edismax</str> > <str name="indent">on</str> > <float name="tie">0.01</float> > </lst> > <lst name="appends"> > <str name="fl">refid</str> > </lst> > <lst name="defaults"> > <str name="echoParams">none</str> > <str name="wt">json</str> > <int name="rows">1000</int> > <str name="qf">smalldesc</str> > <str name="qf">title_text</str> > <str name="qf">titlews^3</str> > <str name="qf">sdescnisq</str> > <str name="qs">1</str> > <str name="mm">2<-1 4<70%</str> > <str name="df">product_titles</str> > </lst> > </requestHandler> > > Response with Cursor and echoParams=all-> *Qtime is now 17* i.e approx 3 > time of previous qtime > responseHeader: { > status: 0, > QTime: 17, > params: { > df: "product_titles", > ps: "3", > echoParams: "all", > indent: "on", > fl: "refid", > tie: "0.01", > defType: "edismax", > qf: "customphonetic", > qs: "1", > qt: "cursor", > sort: "score desc,id asc", > rows: "1000", > q: "bird toy", > cursorMark: "*", > } > }, > response: { > numFound: 17, > start: 0, > docs: [ > { > refid: "152585558" > }, > { > refid: "157276077" > } > ] > } > > > When i curl http://localhost:8080/solr/search/select?q=bird > toy&qt=mapping&ps=3&rows=25&mm=100, i can get results in 3 seconds. > When i curl localhost:8080/solr/search/select?q=bird > toy&qt=cursor&ps=3&rows=1000&mm=100&sort=score desc,id asc&cursorMark=* it > consumed 8 seconds to return result even if the result count=0 > > BTW, the id schema definition is used in sort > <field name="id" type="string" indexed="true" stored="true" required="true" > omitNorms="true" multiValued="false"/> > > Is it due to the sort I have applied or I have implemented it in the wrong > way? > Please help or provide the direction to solve this issue > > > Thanks in advance > > -- > Thanks & Regards, > Ajay Sharma > Product Search > Indiamart Intermesh Ltd. > > -- > >
