[
https://issues.apache.org/jira/browse/LUCENE-7747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15927898#comment-15927898
]
Jim Ferenczi edited comment on LUCENE-7747 at 3/16/17 12:05 PM:
----------------------------------------------------------------
Here is a patch that builds a lazy iterator over the different paths. The graph
boolean query is built by consuming this iterator and throws a "too many
clauses" exception when the number of paths is greater than the max number of
clauses allowed.
was (Author: jim.ferenczi):
Here is a patch that builds an lazy iterator over the different paths. The
graph boolean query is built by consuming this iterator and throws a "too many
clauses" exception when the number of paths is greater than the max number of
clauses allowed.
> QueryBuilder should build side-paths query (graph query) lazily
> ---------------------------------------------------------------
>
> Key: LUCENE-7747
> URL: https://issues.apache.org/jira/browse/LUCENE-7747
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Jim Ferenczi
> Attachments: LUCENE-7747.patch
>
>
> In LUCENE-7638 we generate a query for each multi-token path in the graph and
> combine them at the end in a boolean query.
> This can lead to OOM when the number of path is big, instead we should build
> the disjunction of these paths lazily in order to throw "too many clauses"
> early if the number of paths is bigger than max boolean clauses.
> For instance a shingle filter with shingles of different size produces a
> graph with multiple side paths at each position. If the input query has a lot
> of tokens, the number of paths (query) created is exponential. For this use
> case it is maybe preferable to disallow graph query analysis completely but
> when allowed we should also be protected against combinatorial explosion.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
