Hi, I think i have posted this question in some other thread... When the resultSet is very big, Searching is taking a lot of time. For returning responce of a query that finds approx 14 M results, first time it is taking approx 17Sec. But next time for the same query it is taking almost 2 seconds. I am using Hits for getiing the results searching the result using Searcher.search(). Is there anyother way of improving its speed.
Thanks and regards, On 6/27/06, heritrix. lucene <[EMAIL PROTECTED]> wrote:
No. I am not sorting the data... On 6/27/06, Martin Braun <[EMAIL PROTECTED]> wrote: > > Hi chris, > > searching everytime using a new searcher was taking time. So For > testing, i > > made it a static one and reused the same. This gave me a lot of > > improvement. > > Previously my query was taking approx 25 sec. But now most of the > queries > > are taking time between the 100 and 800 ms. > > Do you use a kind of sorting? I had this issue with SortField.STRING, it > takes a lot of time and memory to build the sort cache of all terms in > the field (of type String). So if the searcher needs to rebuild this > with each query it takes a while... > > hth, > martin > > > Here also i noticed that the query is taking time only first time. If > you > > fire the same query again it takes very very less. > > Can anybody tell me the story behibd this. > > > > With Regards......... > > > > On 6/26/06, Chris Hostetter <[EMAIL PROTECTED]> wrote: > >> > >> > >> : Can you provide some information on your setup? How are you > indexing > >> : and searching? Do you have a lot of terms in your query, > etc? Have > >> you > >> : done any profiling of your setup to determine where the bottlenecks > >> : are? Are you sure they are in Lucene? > >> > >> what methods are you using for doing the search? (Hits, HitCollector, > >> TopDocs) are you sorting? are you opening a new IndexSearcher for > each > >> searcher? what exactly are you timing (a single "search" method > call, an > >> iteration over every matching doc id? an iteration over every > matching > >> doc > >> to get a stored field?) > >> > >> : > >> : -Grant > >> : > >> : heritrix.lucene wrote: > >> : > Hi, > >> : > I have created an index of 47 Million documents. > >> : > I have 1.28GB RAM. > >> : > When i am doing a search over this index it is taking on average > 25 > >> : > sec. Is > >> : > there a way so that i can get results in part of a second... > >> : > I hope there must be some ways.. > >> : > > >> : > Thanks and regards.. > >> : > > >> : > >> : -- > >> : > >> : Grant Ingersoll > >> : Sr. Software Engineer > >> : Center for Natural Language Processing > >> : Syracuse University > >> : School of Information Studies > >> : 335 Hinds Hall > >> : Syracuse, NY 13244 > >> : > >> : http://www.cnlp.org > >> : Voice: 315-443-5484 > >> : Fax: 315-443-6886 > >> : > >> : > >> : > --------------------------------------------------------------------- > >> : To unsubscribe, e-mail: [EMAIL PROTECTED] > >> : For additional commands, e-mail: [EMAIL PROTECTED] > >> : > >> > >> > >> > >> -Hoss > >> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: [EMAIL PROTECTED] > >> For additional commands, e-mail: [EMAIL PROTECTED] > >> > >> > > > > > -- > Universitaetsbibliothek Heidelberg Tel: +49 6221 54-2580 > Ploeck 107-109, D-69117 Heidelberg Fax: +49 6221 54-2623 > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > >
