[
https://issues.apache.org/jira/browse/LUCENE-8058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16263357#comment-16263357
]
Jim Ferenczi commented on LUCENE-8058:
--------------------------------------
I think the problem is emphasized by the fact that
UsageTrackingQueryCachingPolicy caches this query more aggressively.
I agree that this query is not safe for the query cache but we could maybe make
the decision to not cache based on the number of terms ?
This query is already rewritten into a boolean query if there are less than 16
terms so we could use the max boolean clause limit (1024), this way users would
not be tempted to switch to a boolean query to make their big filters eligible
for caching.
> Never cache large TermInSetQuery instances
> ------------------------------------------
>
> Key: LUCENE-8058
> URL: https://issues.apache.org/jira/browse/LUCENE-8058
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Adrien Grand
> Fix For: master (8.0), 7.2
>
> Attachments: LUCENE-8058.patch
>
>
> I have seen several cases in which the query cache was highly underestimating
> its memory usage due to the fact that it had references to large queries that
> ended up using more memory than the associated doc id sets.
> We had a workaround for term-in-set queries by making TermInSetQuery
> implement Accountable, but this information is lost when it is wrapped in
> another query such as a BooleanQuery. So I would like to apply a safer fix
> that just disables caching on large TermInSetQuery instances.
> I know it's a pity given that large queries are probably more expensive and
> thus more cache-worthy, but I see such large queries as the result of a bad
> design or a workaround to the fact that Lucene is not the right tool for the
> job, so I think that disabling caching on large term-in-set queries is the
> right trade-off by making the query cache safer for the majority of our users.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
