Erick, Thanks for the great advice!!
About closing/opening searcher on each request .... isnt this unavoidable in some cases? The application I am building will have users insert/search documents all the time. So for every insert, the searcher need to be recreated again, isnt it? Else new document wont be searchable with the 'old' searcher, right? For this situation, I am thinking of recreating the searcher (and do a warm up) inside the thread that do the insert. With this, the performance penalty occurs to the user that does the insert. Also for my application, there will be more searches than inserts. Is this what people normally do? Thanks. ~KEGan On 9/30/06, Erick Erickson <[EMAIL PROTECTED]> wrote:
Sorting will inevitably have an impact on your speed, but it's impossible to generalize. FWIW, my app has 870K documents, the index is around 1.4G and search/sort times are fine. But even that statement is misleading. "Fine" means that the product manager for this product is satisfied with performance, which has no relevance to your situation <G>...... I'm afraid that you'll just have to put in your sorting and see. I know that's not a very satisfactory answer, but without knowing lots of details about your app AND the distribution of terms AND the expected throughput AND the usage statistics AND......., it's hard to say. It was easy to put together a test harness that fired off a bunch of threads at my searcher and measured throughput. I highly recommend something like this if you're going to try to answer this question before putting the product in production, just so you get an idea of what to expect. Be sure you are satisfied with the performance before adding sorting. Lots of people have gotten into trouble by opening/closing searchers for each request, which is FAR more expensive that sorting in my experience. It would be unfortunate to think your problem was sorting when, in fact, it was something else. Best Erick On 9/29/06, KEGan <[EMAIL PROTECTED]> wrote: > > Erick, > > Ouch!! Please excuse the cut-n-paste ;) > > LIA mentions a lot about performance when doing sorting. Is it something > to > be cautious about? You mention doing 5 fields and it works ok, ... can > share > with us how many documents you are handling there with 5 fields ? > > Thanks. > > ~KEGan > > On 9/29/06, Erick Erickson <[EMAIL PROTECTED]> wrote: > > > > Yes. I do this with 5 fields and it works just fine. Although your > > cut-n-paste got kind of hard to read <G>.... > > > > Erick > > > > On 9/29/06, KEGan <[EMAIL PROTECTED]> wrote: > > > > > > I think I am going to answer my own question. > > > > > > Just use the > > > > > > *Sort*< > > > > > > file:///D:/library/apache/lucene-2.0.0/docs/api/org/apache/lucene/search/Sort.html#Sort(org.apache.lucene.search.SortField[]) > > > > > > > (SortField< > > > > > > file:///D:/library/apache/lucene-2.0.0/docs/api/org/apache/lucene/search/SortField.html > > > > > > > [] fields) > > > *Sort*< > > > > > > file:///D:/library/apache/lucene-2.0.0/docs/api/org/apache/lucene/search/Sort.html#Sort(java.lang.String[]) > > > > > > > (String <http://java.sun.com/j2se/1.4/docs/api/java/lang/String.html > > > > [] fields) > > > > > > This should do it right ? > > > > > > > > > > > > On 9/29/06, KEGan <[EMAIL PROTECTED]> wrote: > > > > > > > > Hi, > > > > > > > > I have seen some sort examples in LIA. But cant find what I am > looking > > > > for. How do I sort document by date, AND for all the documents with > > the > > > same > > > > date ... these are sorted by relavency. (Date has higher sort > priority > > > in > > > > this case). > > > > > > > > Thanks. > > > > > > > > > > > > > > > >
