[
https://issues.apache.org/jira/browse/SOLR-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15392451#comment-15392451
]
Joel Bernstein commented on SOLR-6581:
--------------------------------------
Ok, false alarm. my initial tests were faulty. I thought a had loaded 5,000,000
docs and actually a had set the job to load 50,000,000 docs. So the test was
running with the indexing running.
After running a proper test I found that things are as expected. I'm seeing
to_fc queries running almost 3 times faster with the top_fc hint.
I was running with these simple queries:
{code}
{!collapse field=test_s hint=top_fc}
and
{!collapse field=test_s}
{code}
I had an index of 5 million docs and the test_s field had 1.8 million unique
values.
With the top_fc hint the query was taking around 160 millis.
Without the top_fc hint the query was taking around 440 millis.
> Efficient DocValues support and numeric collapse field implementations for
> Collapse and Expand
> ----------------------------------------------------------------------------------------------
>
> Key: SOLR-6581
> URL: https://issues.apache.org/jira/browse/SOLR-6581
> Project: Solr
> Issue Type: Bug
> Reporter: Joel Bernstein
> Assignee: Joel Bernstein
> Priority: Minor
> Fix For: 5.0, 6.0
>
> Attachments: SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch,
> SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch,
> SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch,
> SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch,
> renames.diff
>
>
> The 4x implementation of the CollapsingQParserPlugin and the ExpandComponent
> are optimized to work with a top level FieldCache. Top level FieldCaches have
> a very fast docID to top-level ordinal lookup. Fast access to the top-level
> ordinals allows for very high performance field collapsing on high
> cardinality fields.
> LUCENE-5666 unified the DocValues and FieldCache api's so that the top level
> FieldCache is no longer in regular use. Instead all top level caches are
> accessed through MultiDocValues.
> This ticket does the following:
> 1) Optimizes Collapse and Expand to use MultiDocValues and makes this the
> default approach when collapsing on String fields
> 2) Provides an option to use a top level FieldCache if the performance of
> MultiDocValues is a blocker. The mechanism for switching to the FieldCache is
> a new "hint" parameter. If the hint parameter is set to "top_fc" then the
> top-level FieldCache would be used for both Collapse and Expand.
> Example syntax:
> {code}
> fq={!collapse field=x hint=TOP_FC}
> {code}
> 3) Adds numeric collapse field implementations.
> 4) Resolves issue SOLR-6066
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
