Sure Todd, the idea basically consist in the following:
- Subclassing FIeldSortedHitQueue and calling support with an empty SortField array: this disables caching because the comparators are retrieved during construction - Creating a new SortComparatorSource that creates the sort comparators similarly to FieldCacheImpl but without performing any caching - Creating a new HitCollector that behaves just like TopFieldDocCollector but uses my custom FieldSortedHitQueue. The tricky part was creating a TopFieldDoc instance (which has default visibility constructor. I managed to do it creating an empty TopFieldDocCollector, asking the TopFieldDocs and modifiying that instance. - Overriding the Searcher's search(Weight, Filter, int, Sort) so it uses my HitCollector (I use the Hits class, which internally calls that method) That way, you completely override field caching and you'll suffer the performance consequences. What I did was making the SortComparatorSource a spring bean and adding a caching interceptor to it. You an plug your favorite cache library in, in my case ehcache. I hope you find this useful. Regards. tbenge wrote: > > Pablo, > > Would you mind adding a little more detail about how you're working > around the problem? > > I'm still evaluating our different options so am interested in what you > did. > > Todd > > On Mon, Nov 3, 2008 at 2:37 PM, PabloS <[EMAIL PROTECTED]> wrote: >> >> Thanks hossman, but I've already 'solved' the problem without the need to >> patch lucene. I had to code a bit around Lucene's visibility restrictions >> but I've managed to completely skip the field caching mechanism and add >> ehcache to it. >> >> At the moment it seems to be working quite well, although not as fast as >> it >> was when lucene performed the caching. >> >> Thanks a lot for the info anyway. >> Regards. >> >> hossman wrote: >>> >>> >>> : I'm having a similar problem with my application, although we are >>> using >>> : lucene 2.3.2. The problem we have is that we are required to sort on >>> most of >>> : the fields (20 at least). Is there any way of changing the cache being >>> used? >>> >>> there is a patch in Jira that takes a completley different approach >>> towards dealing with sorting by using the *stored* values of the >>> documents >>> that match... >>> >>> https://issues.apache.org/jira/browse/LUCENE-769 >>> >>> ...in theory it's "better" for use cases where: >>> >>> 1) you are confident you are going to get back a "small" result set in >>> proportion to the total size of your index. >>> 2) you need to support sorting on "lots" of fields and don't have >>> enough >>> ram for all of the FieldCaches of those fields >>> >>> So far the only person to test it out (and post comments) is the patch >>> submitter, but if other people report success it might be worth adding >>> as >>> a contrib. >>> >>> (Disclaimer: it looks like the patch was updated after my last >>> review/comments. i have not read, nor claim any opinion about the >>> currently attached patch) >>> >>> >>> >>> >>> -Hoss >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [EMAIL PROTECTED] >>> For additional commands, e-mail: [EMAIL PROTECTED] >>> >>> >>> >> >> -- >> View this message in context: >> http://www.nabble.com/OutOfMemory-Problems-Lucene-2.4---Tomcat-tp20236834p20311386.html >> Sent from the Lucene - Java Users mailing list archive at Nabble.com. >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [EMAIL PROTECTED] >> For additional commands, e-mail: [EMAIL PROTECTED] >> >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > > -- View this message in context: http://www.nabble.com/OutOfMemory-Problems-Lucene-2.4---Tomcat-tp20236834p20326783.html Sent from the Lucene - Java Users mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
