Re: Lucene TermsFilter lookup slow

2015-08-18 Thread jamie
Michael Forgive me, I am not familiar with Lucene internal code. Can you verify whether these suggested changes are indeed correct. I am changing line 210 of TermsFilter. if (result == null) { if (docs.nextDoc() != DocIdSetIterator.NO_MORE_DOCS) { result = new FixedBitSet(re

Re: Lucene TermsFilter lookup slow

2015-08-10 Thread Michael McCandless
OK, indeed, that version has the changes I was thinking of, specifically optimizing the case when only a single doc contains a term by inlining that docID into the terms dict. You should be able to improve on TermsFilter a bit because you know only 1 doc matches each ID, so after the first segment

Re: Lucene TermsFilter lookup slow

2015-08-09 Thread jamie
Mike Thank you kindly for the reply. I am using Lucene v4.10.4. Are the optimization you refer to, available in this version? We haven't yet upgraded to Lucene 5 as there appear to be many API changes. Jamie On 2015/08/08 5:13 PM, Michael McCandless wrote: Which version of Lucene are you us

Re: Lucene TermsFilter lookup slow

2015-08-08 Thread Michael McCandless
Which version of Lucene are you using? Newer versions have optimized the "primary key" use case somewhat... Mike McCandless http://blog.mikemccandless.com On Sat, Aug 8, 2015 at 8:32 AM, jamie wrote: > Greetings > > Our app primarily uses Lucene for its intended purpose i.e. to search across