Keith Laban created SOLR-8559:
---------------------------------
Summary: FCS facet performance optimization
Key: SOLR-8559
URL: https://issues.apache.org/jira/browse/SOLR-8559
Project: Solr
Issue Type: Bug
Reporter: Keith Laban
While profiling a large collection (multi-sharded billions of documents), I
found that a fast (5-10ms query) would take 20-30 seconds when doing facets
even when {{facet.mincount=1}}
Profiling made it apparent that with {{facet.method=fcs}} 99% of the time was
[spent
here|https://github.com/apache/lucene-solr/blob/trunk/solr/core/src/java/org/apache/solr/request/PerSegmentSingleValuedFaceting.java#L212].
{{queue.udpateTop}} gets called {{numOfSegments*numTerms}}, the worst case when
every term is in every segment. This formula doesn't take into account whether
or not any of the terms have a positive count with respect to the docset.
These optimizations are aimed to do two things:
# When mincount>0 don't include segments which all terms have zero counts. This
should significantly speed up processing when terms are high cardinality and
the matching docset is small
# FIXED TODO optimization: when mincount>0 move segment position the next non
zero term value.
both of these changes will minimize the number of called needed to the slow
{{updateTop}} call.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
