Adrien Grand created LUCENE-6032:
------------------------------------
Summary: Dealing with slow iterators
Key: LUCENE-6032
URL: https://issues.apache.org/jira/browse/LUCENE-6032
Project: Lucene - Core
Issue Type: Wish
Reporter: Adrien Grand
Priority: Minor
This is a recurring issue (for instance already discussed in LUCENE-5418) but
queries can sometimes be super slow if they wrap a filter that provides
linear-time nextDoc/advance.
LUCENE-5418 has the following comment:
bq. New patch, throwing UOE from DocIdSet.iterator() for the Filter returned by
Range.getFilter(). I like this approach: it's safer for the user so they don't
accidentally apply a super slow filter.
I like this approach because doc id sets not providing efficient iteration
should really be an exception rather than a common case. In addition, using an
exception has the benefit of propagating the information through the call
stack, which would not be the case if we used null or a sentinel value to say
that the iterator is super slow. So if you write a filter that can wrap other
filters and doesn't know how to deal with filters that don't support efficient
iteration, you do not need to modify your code: it will work just fine with
filters that support fast iteration and will fail on filters that don't.
Something I would like to explore is whether things like FilteredQuery could
catch this exception in order to fall back automatically to a random-access
strategy?
The general idea I have is that it is ok to apply a random filter as long as
you have a fast iterator to drive iteration? So eg. a filtered query based on a
slow iterator would make sense, but not a ConstantScoreQuery that would wrap a
filter since it would need to evaluate the filter on all non-deleted documents
(it would propagate the exception of the filter).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
