Re: Solr facets counts deep paged returns inconsistent counts

2017-10-20 Thread Yonik Seeley
On Fri, Oct 20, 2017 at 2:22 PM, kenny wrote: > Thanks for the clear explanation. A couple of follow up questions > > - can we tune overrequesting in json API? > Yes, I still need to document it, but you can specify a specific number of documents to overrequest: { type :

Re: Solr facets counts deep paged returns inconsistent counts

2017-10-20 Thread kenny
Thanks for the clear explanation. A couple of follow up questions - can we tune overrequesting in json API? - we do see conflicting counts but that's when we have offsets different from 0. We have now already tested it in solr 6.6 with json api. We sometimes get the same value in different

Re: Solr facets counts deep paged returns inconsistent counts

2017-10-20 Thread Yonik Seeley
Facet refinement in Solr guarantees that counts for returned constraints are correct, but does not guarantee that the top N returned isn't missing a constraint. Consider the following shard counts (3 shards) for the following constraints (aka facet values): constraintA: 2 0 0 constraintB: 0 2 0

Solr facets counts deep paged returns inconsistent counts

2017-10-20 Thread kenny
Hi all, When we run some 'deep' facet counts (eg facet values from 0 to 500 and then from 500 to 1000), we see small but disturbing difference in counts between the two (for example last count on first batch 165, first count on second batch 167) We run this on solr 5.3.1 in cloud mode (3