[
https://issues.apache.org/jira/browse/SOLR-2908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13903956#comment-13903956
]
Shalin Shekhar Mangar commented on SOLR-2908:
---------------------------------------------
Perhaps incorrect counts are okay sometimes? We have the same problem with the
SpellCheckComponent and the FacetComponent. Both of them push a limit to the
shards which is slightly higher than the specified limit during distributed
search. None of them guarantee accurate values unless you set the limit to -1
(unlimited).
Instead of a 'shards.terms.params.override' parameter, perhaps we can have a
'terms.shard.limit' parameter which can default to a sane limit, say
terms.limit * 1.5 + 10 (this formula is used by faceting). If necessary the
user can specify a higher value if accuracy is desired. What do you think?
> To push the terms.limit parameter from the master core to all the shard cores.
> ------------------------------------------------------------------------------
>
> Key: SOLR-2908
> URL: https://issues.apache.org/jira/browse/SOLR-2908
> Project: Solr
> Issue Type: Improvement
> Components: SearchComponents - other
> Affects Versions: 1.4.1
> Environment: Linux server. 64 bit processor and 16GB Ram.
> Reporter: sivaganesh
> Priority: Critical
> Labels: patch
> Fix For: 4.7
>
> Attachments: SOLR-2908.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> When we pass the terms.limit parameter to the master (which has many shard
> cores), it's not getting pushed down to the individual cores. Instead the
> default value of -1 is assigned to Terms.limit parameter is assigned in the
> underlying shard cores. The issue being the time taken by the Master core to
> return the required limit of terms is higher when we are having more number
> of underlying shard cores. This affects the performances of the auto suggest
> feature.
> Can thought we can have a parameter to explicitly override the -1 being set
> to Terms.limit in shards core.
> We saw the source code(TermsComponent.java) and concluded that the same.
> Please help us in pushing the terms.limit parameter to shard cores.
> PFB code snippet.
> private ShardRequest createShardQuery(SolrParams params) {
> ShardRequest sreq = new ShardRequest();
> sreq.purpose = ShardRequest.PURPOSE_GET_TERMS;
> // base shard request on original parameters
> sreq.params = new ModifiableSolrParams(params);
> // remove any limits for shards, we want them to return all possible
> // responses
> // we want this so we can calculate the correct counts
> // dont sort by count to avoid that unnecessary overhead on the shards
> sreq.params.remove(TermsParams.TERMS_MAXCOUNT);
> sreq.params.remove(TermsParams.TERMS_MINCOUNT);
> sreq.params.set(TermsParams.TERMS_LIMIT, -1);
> sreq.params.set(TermsParams.TERMS_SORT, TermsParams.TERMS_SORT_INDEX);
> return sreq;
> }
> Solr Version:
> Solr Specification Version: 1.4.0.2010.01.13.08.09.44
> Solr Implementation Version: 1.5-dev exported - yonik - 2010-01-13 08:09:44
> Lucene Specification Version: 2.9.1-dev
> Lucene Implementation Version: 2.9.1-dev 888785 - 2009-12-09 18:03:31
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
