[
https://issues.apache.org/jira/browse/LUCENE-10103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17428222#comment-17428222
]
Michael McCandless commented on LUCENE-10103:
---------------------------------------------
Thanks [~zhai7631] – I just pushed the PR to main. Is there any reason not to
backport to 8.11? Until the 9.0 bits are actually set free, flying away, I
think we should backport all changes that we can (do not break back compat,
etc.).
Though, perhaps this change is such a big change in behavior that it should
remain 9.0 only? Or I could see it as a bug-fix ...
> QueryCache not estimating query size properly
> ---------------------------------------------
>
> Key: LUCENE-10103
> URL: https://issues.apache.org/jira/browse/LUCENE-10103
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/search
> Reporter: Haoyu Zhai
> Priority: Minor
> Attachments: query_cache_error_demo.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> QueryCache seems estimating the cached query size using a
> [constant|https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/search/LRUQueryCache.java#L302],
> it will cause OOM error in some extreme cases where queries cached will use
> far more memories than assumed. (The default QueryCache tries to use [only 5%
> of
> heap|https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/search/IndexSearcher.java#L89])
> One example of such memory-eating query is AutomatonQuery, it will each
> carry a
> [RunAutomaton|https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/util/automaton/RunAutomaton.java#L42]
> , which consumes a good amount of memory in exchange for the speed.
> On the other hand, we actually have a good implementation of {{Accountable}}
> interface for AutomatonQuery (though it will become a bit more complicated
> later since this query will eventually be rewritten to something else), so
> maybe QueryCache could use those estimation directly (using an {{instanceof}}
> check)? Or moreover we could make all {{Query}} implement {Accountable}}, and
> maybe the default implementation could just be returning the current constant
> we're using, and only override the method of the potential troublesome
> queries?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
