It would be interesting to see the results of using a custom IndexReader that implements http://dsiutils.dsi.unimi.it/docs/it/unimi/dsi/util/ImmutableExternalPrefixMap.htmlor something like it. The only problem right now would be hooking into the Lucene SegmentMerger to merge other indices such as a specialized prefix index.
On Thu, May 22, 2008 at 1:22 PM, Otis Gospodnetic < [EMAIL PROTECTED]> wrote: > Some quick feedback. Those are all very expensive queries (wildcards and > ranges). The first thing I'd do is try without Hibernate Search (to make > sure HS is not the bottleneck). 100 threads is a lot, I'm guessing you are > reusing your searcher, which is good, but you will actually improve > performance a bit if you work with a small pool of searchers instead of a > single searcher. > > Otis > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > ----- Original Message ---- > > From: Rakesh Shete <[EMAIL PROTECTED]> > > To: [EMAIL PROTECTED]; java-user@lucene.apache.org > > Sent: Thursday, May 22, 2008 1:16:13 PM > > Subject: Improving search performance > > > > > > Hi all, > > > > I have index of size 85MB. My query looks as follows: > > > > +(t:boss* d:boss* dd:boss* tg:boss*) +st:act +ntid:0 +cid:1 +dr:[20080410 > TO > > 20081010] +rT:[002 TO 005] > > > > All the fields used in the query are stored in the indexes (Indexed & > Stored) > > > > The query response time for me is around 30 seconds when running mutliple > > simultanoeous threads (~100). The no. of matches is ~30k but I retrieve > only the > > top 100 results. I am using Hibernate Search which is a wrapper around > Lucene. I > > retrieve the "id" filed from the index which is also indexex and stored. > > > > What is the approach that I should take for improving the performance? > > > > Will just indexing the values without storing them work (Index & > UnStored)? > > > > My machine configuration is: > > P4 2.66GHz 1.99 GB RAM > > > > The code for searching runs in JBoss application server which has a > maximum heap > > size of 1024MB. When these 100 threads are running in the application > server the > > CPU utilization is 100% and JBoss consumes all of the heap size. > > > > Any pointers on index optimization would be really appreciated. > > > > --Regards, > > Rakesh Shete > > > > _________________________________________________________________ > > No Harvard, No Oxford. We are here. Find out !! > > http://ss1.richmedia.in/recurl.asp?pid=500 > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > >
