Re: possible TermInfosReader speedup

2009-04-09 Thread Mike Klaas
On 8-Apr-09, at 11:13 PM, Michael Busch wrote: I was thinking about doing this as part of LUCENE-1195. However, I doubt that the net win will be very noticeable here. A common scenario is that you have an index with one big body field that has a lot of unique terms, plus several metafield

Re: possible TermInfosReader speedup

2009-04-09 Thread Michael McCandless
OK I opened https://issues.apache.org/jira/browse/LUCENE-1592. Mike On Thu, Apr 9, 2009 at 4:36 AM, Michael McCandless wrote: > On Thu, Apr 9, 2009 at 4:24 AM, Uwe Schindler wrote: > >> I think, we should do what was suggested in this thread: Remove it or >> deprecate it, if it is nowhere used

Re: possible TermInfosReader speedup

2009-04-09 Thread Michael McCandless
On Thu, Apr 9, 2009 at 4:24 AM, Uwe Schindler wrote: > I think, we should do what was suggested in this thread: Remove it or > deprecate it, if it is nowhere used internally to prevent people (like me in > the past) to try to use it. > > Maybe put an additional warning in the JavaDocs in addition

RE: possible TermInfosReader speedup

2009-04-09 Thread Uwe Schindler
> > Yes, if skipTo would work more performant, I could easily use it in > > TrieRange and would be happy as noted before. Currently, a new TermEnum > is > > created on each sub-range. When TrieRange was committed and therefore > > updated, for me it was (and still is) not clear, why skipTo may not

Re: possible TermInfosReader speedup

2009-04-08 Thread Michael Busch
On 4/8/09 2:08 PM, Earwin Burrfoot wrote: On Thu, Apr 9, 2009 at 00:14, Michael McCandless wrote: On Wed, Apr 8, 2009 at 3:46 PM, Earwin Burrfoot wrote: Currently, when we're seeking a given Term, it does a binary search across all term space, including terms belonging to other fi

Re: possible TermInfosReader speedup

2009-04-08 Thread Earwin Burrfoot
On Thu, Apr 9, 2009 at 02:01, Uwe Schindler wrote: >> >> Also, on the other topic - how hard is it to boost >> >> TermEnum.skipTo(term) speed to IndexReader.terms(term) level? Would be >> >> nice for TrieRangeFilter and probably some other filters. >> > I think all that's needed is to implement Se

RE: possible TermInfosReader speedup

2009-04-08 Thread Uwe Schindler
> >> Also, on the other topic - how hard is it to boost > >> TermEnum.skipTo(term) speed to IndexReader.terms(term) level? Would be > >> nice for TrieRangeFilter and probably some other filters. > > I think all that's needed is to implement SegmentTermEnum.skipTo, > > calling something like tis.ter

Re: possible TermInfosReader speedup

2009-04-08 Thread Earwin Burrfoot
On Thu, Apr 9, 2009 at 00:14, Michael McCandless wrote: > On Wed, Apr 8, 2009 at 3:46 PM, Earwin Burrfoot wrote: > >> Currently, when we're seeking a given Term, it does a binary search >> across all term space, including terms belonging to other fields. >> I propose augmenting fields file with t

Re: possible TermInfosReader speedup

2009-04-08 Thread Michael McCandless
On Wed, Apr 8, 2009 at 3:46 PM, Earwin Burrfoot wrote: > Currently, when we're seeking a given Term, it does a binary search > across all term space, including terms belonging to other fields. > I propose augmenting fields file with two pointers (firstTerm, > lastTerm) for each field. That reduce

possible TermInfosReader speedup

2009-04-08 Thread Earwin Burrfoot
Currently, when we're seeking a given Term, it does a binary search across all term space, including terms belonging to other fields. I propose augmenting fields file with two pointers (firstTerm, lastTerm) for each field. That reduces range we need to search, and instead of comparing Terms we only