[
https://issues.apache.org/jira/browse/LUCENE-5917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14118179#comment-14118179
]
Michael McCandless commented on LUCENE-5917:
--------------------------------------------
Unfortunately such adversarial queries do exist in Lucene ... it's not clear
how best to address this.
Adding Accountable to Query is a little tricky because it depends on the
searcher (Accountable doesn't take searcher today) and for these MTQs ... they
have to do quite a bit of work in rewrite to figure out which strategy they
will use in order to figure out how much RAM that strategy will take.
Also, with LUCENE-5879, we may be able to reduce the times that we write to the
bitset for prefix queries.
> complex Query cause luene outMemory
> -----------------------------------
>
> Key: LUCENE-5917
> URL: https://issues.apache.org/jira/browse/LUCENE-5917
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/search
> Affects Versions: 4.9
> Environment: 256G RAM + 1.5T SSD + 1.2T DATA(10*10000*10000 records).
> java heap=128G, G1 GC
> Reporter: Littlestar
> Priority: Minor
>
> RangeQuery, prefixQuery and WildcardQuery use FixedBitSet when TERM_COUNT >=
> 350 or DOC_COUNT_PERCENT >=0.1.
> It use a lots of memory when maxDoc very large.
> MultiTermQueryWrapperFilter<Q extends MultiTermQuery> extends Filter
> a little threads run with query "a* OR b* OR c*.....OR z*“ will cause luene
> outMemory, but there is no ways to prevent it.
> another thing, some complex query also use a lot of memory too..
> I think query implements Accountable(#ramSizeInBytes, #ramBytesUsed), users
> can throw a exception better than OutOfMemory.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
