Great - this issue? https://issues.apache.org/jira/browse/LUCENE-2127
Sounds like it would be a real win for lucene. -Peter On Thu, Jan 7, 2010 at 4:12 PM, Otis Gospodnetic <otis_gospodne...@yahoo.com> wrote: > Peter - Aaron just commented on a recent Solr issue (reading large result > sets) and mentioned his patch. > So far he has 2 x +1 from Grant and me to stick his patch in JIRA. > > Otis > -- > Sematext -- http://sematext.com/ -- Solr - Lucene - Nutch > > > > ----- Original Message ---- >> From: Peter Wolanin <peter.wola...@acquia.com> >> To: solr-user@lucene.apache.org >> Sent: Sun, January 3, 2010 3:37:01 PM >> Subject: Re: SOLR Performance Tuning: Pagination >> >> At the NOVA Apache Lucene/Solr Meetup last May, one of the speakers >> from Near Infinity (Aaron McCurry I think) mentioned that he had a >> patch for lucene that enabled unlimited depth memory-efficient paging. >> Is anyone in contact with him? >> >> -Peter >> >> On Thu, Dec 24, 2009 at 11:27 AM, Grant Ingersoll wrote: >> > >> > On Dec 24, 2009, at 11:09 AM, Fuad Efendi wrote: >> > >> >> I used pagination for a while till found this... >> >> >> >> >> >> I have filtered query ID:[* TO *] returning 20 millions results (no >> >> faceting), and pagination always seemed to be fast. However, fast only >> >> with >> >> low values for start=12345. Queries like start=28838540 take 40-60 >> >> seconds, >> >> and even cause OutOfMemoryException. >> > >> > Yeah, deep pagination in Lucene/Solr can be problematic due to the Priority >> Queue management. See http://issues.apache.org/jira/browse/LUCENE-2127 and >> the >> linked discussion on java-dev. >> > >> >> >> >> I use highlight, faceting on nontokenized "Country" field, standard >> >> handler. >> >> >> >> >> >> It even seems to be a bug... >> >> >> >> >> >> Fuad Efendi >> >> +1 416-993-2060 >> >> http://www.linkedin.com/in/liferay >> >> >> >> Tokenizer Inc. >> >> http://www.tokenizer.ca/ >> >> Data Mining, Vertical Search >> >> >> >> >> >> >> >> >> > >> > -------------------------- >> > Grant Ingersoll >> > http://www.lucidimagination.com/ >> > >> > Search the Lucene ecosystem using Solr/Lucene: >> http://www.lucidimagination.com/search >> > >> > >> >> >> >> -- >> Peter M. Wolanin, Ph.D. >> Momentum Specialist, Acquia. Inc. >> peter.wola...@acquia.com > > -- Peter M. Wolanin, Ph.D. Momentum Specialist, Acquia. Inc. peter.wola...@acquia.com
