[
https://issues.apache.org/jira/browse/SOLR-8096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15065639#comment-15065639
]
Jamie Johnson commented on SOLR-8096:
-------------------------------------
Sorry if this is the incorrect place for this, but I took a stab at trying to
implement supporting uninverted field based facets in SimpleFacets. i am not
sure it's 100% there but I think it's close. The basic approach was to
leverage as much as possible from the JSON Faceting API since that is the only
consumer of the UIF that I could find. This meant I had to make some classes
public that were previously package protected (Perhaps moving SimpleFacets into
the facet package would have been better?). Additionally, I had to make
FacetProcessor aware of Grouping so that the docset would be adjusted
appropriately for grouping requests with truncate set to true.
Also I did this by adding DV as a new FacetMethod and made it so this is what
triggers using DocValues vs FC which currently triggers it. Perhaps it would
be more appropriate to add a new FacetMethod named UIF and leave FC alone? I'm
open to suggestions here.
Last significant difference from the 4.10.4 implementation is I didn't attempt
to use the Lucene FieldCache at all since it was made package protected. The
4.10.4 implementation used that in cases, but this should be inline with what
JSON Facets is doing.
The commits are attached as a patch to this ticket (I'm happy to spawn off a
new ticket if it's more appropriate) and also available at
https://github.com/jej2003/lucene-solr
> Major faceting performance regressions
> --------------------------------------
>
> Key: SOLR-8096
> URL: https://issues.apache.org/jira/browse/SOLR-8096
> Project: Solr
> Issue Type: Bug
> Affects Versions: 5.0, 5.1, 5.2, 5.3, Trunk
> Reporter: Yonik Seeley
> Priority: Critical
>
> Use of the highly optimized faceting that Solr had for multi-valued fields
> over relatively static indexes was removed as part of LUCENE-5666, causing
> severe performance regressions.
> Here are some quick benchmarks to gauge the damage, on a 5M document index,
> with each field having between 0 and 5 values per document. *Higher numbers
> represent worse 5x performance*.
> Solr 5.4_dev faceting time as a percent of Solr 4.10.3 faceting time
> ||...................................|| Percent of index being faceted
> ||num_unique_values|| 10% || 50% || 90% ||
> |10 | 351.17% | 1587.08% | 3057.28% |
> |100 | 158.10% | 203.61% | 1421.93% |
> |1000 | 143.78% | 168.01% | 1325.87% |
> |10000 | 137.98% | 175.31% | 1233.97% |
> |100000 | 142.98% | 159.42% | 1252.45% |
> |1000000 | 255.15% | 165.17% | 1236.75% |
> For example, a field with 1000 unique values in the whole index, faceting
> with 5x took 143% of the 4x time, when ~10% of the docs in the index were
> faceted.
> One user who brought the performance problem to our attention:
> http://markmail.org/message/ekmqh4ocbkwxv3we
> "faceting is unusable slow since upgrade to 5.3.0" (from 4.10.3)
> The disabling of the UnInvertedField algorithm was previously discovered in
> SOLR-7190, but we didn't know just how bad the problem was at that time.
> edit: removed "secret" adverb by request
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
