Hi Ivan! btw may be forbidding the sorted search in case of too many results is an option? I did this way in my case.
Regards, Artem. On 4/24/07, Artem Vasiliev <[EMAIL PROTECTED]> wrote:
Ahhh, you said in your original post that your search matches _all_ the results.. Yup my patch will not help much in this case - after all all the values have to be read to be compared while sorting! :) LUCENE-769 patch helps only if result set is significantly less than full index size. Regards, Artem On 4/24/07, Artem Vasiliev <[EMAIL PROTECTED]> wrote: > > Hello Ivan! > > It's so sad to me that you had bad results with that patch. :) > > The discussion in the ticket is out-of-date - the patch was initially in > several classes, used WeakHashMap but then it evolved to what it's now - one > StoredFieldSortFactory class. I use it in my sharehound app in pretty much > the same the form it is in Jira currently and it does show good results to > me. > > In your sample searches, > - how many results do you have? > - how long does the sorted search execute? > - what is the average size of a sorted field? > - what is the CPU and how much of it and memory you give to the > application? > > I get page 1 (first 100 items) of sorted list with 10000 items in 0.3sto 3s (for date column it exactly depends on whether the sort is ascending > or descending - don't know why is that). My index is about 1mln docs and 1G; > sorted fields are rather small (numbers, dates and string of maybe 50 > symbols average). The machine looks quite beefy to me - Intel core duo with > 500M given to the application. > > Regards, > Artem > > On 4/23/07, Ivan Vasilev < [EMAIL PROTECTED]> wrote: > > > > Hi All, > > THANK YOU FOR YOUR HELP :) > > I put this problem in the forum but I had no chance to work on it last > > week unfurtunately... > > So now I tested the Artem's patch but the results show: > > 1) speed is very slow compare with the usage without patch > > 2) There are not very big differences of memory usage (I tested till > > now > > only with relativly small indexes - less than 1 GB and less than 1 mil > > docs because the when using with 20-40 GB indexes I had to wait more > > than 5 mins what is practically usless). > > > > So I have doubts if I use the patch correctly. I do just what is > > described in Artem's letter: > > > > AV> You can include StoredFieldSortFactory class source file into your > > sources and > > AV> then use StoredFieldSortFactory.create(sortFieldName, > > sortDescending) to get > > AV> Sort object for sorting query. > > AV> StoredFieldSortFactory source file can be extracted from > > LUCENE-769 patch or > > AV> from sharehound sources: http://sharehound.cvs.sourceforge.net/*checkout*/sharehound/jNetCrawler/src/java/org/apache/lucene/search/StoredFieldSortFactory.java > > > > > > > > What I am wondering about is that in the patch commetns > > (https://issues.apache.org/jira/browse/LUCENE-769 ) I see that there > > is > > written that patch solves the problem by using WeakHashMap, but > > actually > > in the downloaded StoredFieldSortFactory.java file there is not used > > WeakHashMap. Another thing: In the comments in Lucene-769 issue there > > is > > mentioned something about classes like: WeakDocumentsCache and > > DocCachingIndexReader but I did not found them in Lucene source code > > neither as classes in StoredFieldSortFactory.java. So my questions > > are: > > 1. Is it enought to include the file StoredFieldSortFactory.java in > > the > > source code or there are also other classes that I have to douwnload > > and > > include? > > 2. Have I to use this DocCachingIndexReader instead of Reader that I > > currently use in cases when I expect OOMException and will use this > > patch? > > > > Thanks to all once again :), > > Ivan > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > >
