[ https://issues.apache.org/jira/browse/LUCENE-7747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15927898#comment-15927898 ]
Jim Ferenczi edited comment on LUCENE-7747 at 3/16/17 12:05 PM: ---------------------------------------------------------------- Here is a patch that builds a lazy iterator over the different paths. The graph boolean query is built by consuming this iterator and throws a "too many clauses" exception when the number of paths is greater than the max number of clauses allowed. was (Author: jim.ferenczi): Here is a patch that builds an lazy iterator over the different paths. The graph boolean query is built by consuming this iterator and throws a "too many clauses" exception when the number of paths is greater than the max number of clauses allowed. > QueryBuilder should build side-paths query (graph query) lazily > --------------------------------------------------------------- > > Key: LUCENE-7747 > URL: https://issues.apache.org/jira/browse/LUCENE-7747 > Project: Lucene - Core > Issue Type: Improvement > Reporter: Jim Ferenczi > Attachments: LUCENE-7747.patch > > > In LUCENE-7638 we generate a query for each multi-token path in the graph and > combine them at the end in a boolean query. > This can lead to OOM when the number of path is big, instead we should build > the disjunction of these paths lazily in order to throw "too many clauses" > early if the number of paths is bigger than max boolean clauses. > For instance a shingle filter with shingles of different size produces a > graph with multiple side paths at each position. If the input query has a lot > of tokens, the number of paths (query) created is exponential. For this use > case it is maybe preferable to disallow graph query analysis completely but > when allowed we should also be protected against combinatorial explosion. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org