[
https://issues.apache.org/jira/browse/SOLR-16693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17725557#comment-17725557
]
David Smiley commented on SOLR-16693:
-------------------------------------
Unfortunately retaining the old behavior would add a *new* performance penalty
for everyone *not* using timeAllowed due to Lucene's design change :-/. So I'm
definitely not in favor of continuing with ExitableDirectoryReader unless maybe
if it could be optional per-query. Maybe it could.
First, let's explore what does the behavior look like before this change for
such a feature? BTW there is a test "ExitableDirectoryReaderTest" in Solr that
perhaps should be named "TimeAllowedTest" instead of the implementation detail
of how it's implemented. It doesn't use faceting but maybe it should? And/or
spellcheck? We might end up adding a check before each faceting step (e.g.
between fields) instead of doing a slow term-by-term check.
> Use TimeLimitingBulkScorer; stop using ExitableDirectoryReader
> --------------------------------------------------------------
>
> Key: SOLR-16693
> URL: https://issues.apache.org/jira/browse/SOLR-16693
> Project: Solr
> Issue Type: Improvement
> Affects Versions: 9.2
> Reporter: David Smiley
> Assignee: David Smiley
> Priority: Blocker
> Fix For: 9.3
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> While reviewing Lucene 9.5 changes coming into Solr, I noticed some changes
> relating to the ability to specify timeAllowed in Solr on a search query.
> Solr uses both ExitableDirectoryReader and TimeLimitingCollector from Lucene
> for this (complementary to each other). Unfortunately, changes in Lucene
> will make the cost of ExitableDirectoryReader wrapping happen for all queries
> into Solr, even those not using timeAllowed. Options to keep EDR aren't good
> -- fork it basically. Anecdotally, I think I've heard the overhead is not
> trivial and my intuition thinks likewise. Meanwhile, Lucene 9.3 added a new
> TimeLimitingBulkScorer which even gets first class integration into
> IndexSearcher which has a timeout. It's been incrementally improved, and I
> really like its approach, probable performance, and simplicity. It should be
> straightforward to integrate this into SolrIndexSearcher and also only do so
> for queries specifying timeAllowed. I'm not sure TimeLimitingCollector
> offers much value to using TLBS other than additional precision on
> timeAllowed at some cost to unselective queries.
> I think doing this should block Solr 9.2 using Lucene 9.5. Alternatively,
> someone might benchmark the state of things and see that things aren't so bad
> as they may seem. But that takes work too.
> [1] QueryTimeout.isTimeoutEnabled is gone:
> https://github.com/apache/lucene/pull/11954
> [2] TimeLimitingBulkScorer in LUCENE-10151
> https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/search/TimeLimitingBulkScorer.java
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
