[
https://issues.apache.org/jira/browse/LUCENE-5917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14117852#comment-14117852
]
Littlestar commented on LUCENE-5917:
------------------------------------
{code}
MultiTermQueryWrapperFilter.java
...........
@Override
public DocIdSet getDocIdSet(AtomicReaderContext context, Bits acceptDocs)
throws IOException {
final AtomicReader reader = context.reader();
final Fields fields = reader.fields();
if (fields == null) {
// reader has no fields
return null;
}
final Terms terms = fields.terms(query.field);
if (terms == null) {
// field does not exist
return null;
}
final TermsEnum termsEnum = query.getTermsEnum(terms);
assert termsEnum != null;
if (termsEnum.next() != null) {
// fill into a FixedBitSet
final FixedBitSet bitSet = new FixedBitSet(context.reader().maxDoc());
//==================here......
DocsEnum docsEnum = null;
do {
// System.out.println(" iter termCount=" + termCount + " term=" +
// enumerator.term().toBytesString());
docsEnum = termsEnum.docs(acceptDocs, docsEnum, DocsEnum.FLAG_NONE);
int docid;
while ((docid = docsEnum.nextDoc()) != DocIdSetIterator.NO_MORE_DOCS) {
bitSet.set(docid);
}
} while (termsEnum.next() != null);
// System.out.println(" done termCount=" + termCount);
return bitSet;
} else {
return null;
}
}
{code}
> complex Query cause luene outMemory
> -----------------------------------
>
> Key: LUCENE-5917
> URL: https://issues.apache.org/jira/browse/LUCENE-5917
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/search
> Affects Versions: 4.9
> Environment: 128G RAM + 1.5T SSD + 1.2T DATA(10*10000*10000 records).
> Reporter: Littlestar
> Priority: Minor
>
> RangeQuery, prefixQuery and WildcardQuery use FixedBitSet when TERM_COUNT >=
> 350 or DOC_COUNT_PERCENT >=0.1.
> It use a lots of memory when maxDoc very large.
> MultiTermQueryWrapperFilter<Q extends MultiTermQuery> extends Filter
> a little threads run with query "a* OR b* OR c*.....OR z*“ will cause luene
> outMemory, but there is no ways to prevent it.
> another thing, some complex query also use a lot of memory too..
> I think query implements Accountable(#ramSizeInBytes), users can throw a
> exception better than OutOfMemory.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
