[
https://issues.apache.org/jira/browse/SOLR-11159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16103359#comment-16103359
]
Amrit Sarkar commented on SOLR-11159:
-------------------------------------
Some of my observations from debugQuery,
Here's how per shard request is made for various purposes COUNT DEC, LIMIT 2 :
1st request:
{code}
"Response":"{responseHeader={zkConnected=true,status=0,QTime=2826,params={df=_text_,distrib=false,debug=[false,
timing, track],_facet_={},fl=[id,
score],shards.purpose=1048580,start=0,fsv=true,shard.url=http://127.0.0.1:8983/solr/collection1_shard3_replica_n4/,rows=10,rid=127.0.0.1-collection1_shard3_replica_n4-1501147655592-0,version=2,q=*:*,json.facet={cat_s:{type:terms,field:cat_s,sort:\"count
desc\",limit:2,overrequest:0,refine:true}},requestPurpose=GET_TOP_IDS,NOW=1501147655590,isShard=true,wt=javabin,debugQuery=false}},response={numFound=1,start=0,maxScore=1.0,docs=[SolrDocument{id=5,
score=1.0}]},sort_values={},facets={count=1,cat_s={buckets=[{val=C,count=1}]}},debug={facet-trace={processor=FacetQueryProcessor,elapse=2644,query=null,domainSize=1,sub-facet=[{processor=FacetFieldProcessorByArrayDV,elapse=1632,field=cat_s,limit=2,numBuckets=1,domainSize=1}]},json={facet={cat_s={type=terms,
field=cat_s, sort=count desc, limit=2, overrequest=0,
refine=true}}},timing={time=2817.0,prepare={time=7.0,query={time=1.0},facet={time=0.0},facet_module={time=6.0},mlt={time=0.0},highlight={time=0.0},stats={time=0.0},expand={time=0.0},terms={time=0.0},debug={time=0.0}},process={time=2741.0,query={time=0.0},facet={time=0.0},facet_module={time=2665.0},mlt={time=0.0},highlight={time=0.0},stats={time=0.0},expand={time=0.0},terms={time=0.0},debug={time=0.0}}}}}"},
{code}
2nd request:
{code}
"Response":"{responseHeader={zkConnected=true,status=0,QTime=2828,params={df=_text_,distrib=false,debug=[false,
timing, track],_facet_={},fl=[id,
score],shards.purpose=1048580,start=0,fsv=true,shard.url=http://127.0.0.1:8983/solr/collection1_shard2_replica_n2/,rows=10,rid=127.0.0.1-collection1_shard3_replica_n4-1501147655592-0,version=2,q=*:*,json.facet={cat_s:{type:terms,field:cat_s,sort:\"count
desc\",limit:2,overrequest:0,refine:true}},requestPurpose=GET_TOP_IDS,NOW=1501147655590,isShard=true,wt=javabin,debugQuery=false}},response={numFound=7,start=0,maxScore=1.0,docs=[SolrDocument{id=2,
score=1.0}, SolrDocument{id=4, score=1.0}, SolrDocument{id=3, score=1.0},
SolrDocument{id=6, score=1.0}, SolrDocument{id=9, score=1.0},
SolrDocument{id=12, score=1.0}, SolrDocument{id=15,
score=1.0}]},sort_values={},facets={count=7,cat_s={buckets=[{val=C,count=4},
{val=A,count=1}]}},debug={facet-trace={processor=FacetQueryProcessor,elapse=2090,query=null,domainSize=7,sub-facet=[{processor=FacetFieldProcessorByArrayDV,elapse=1098,field=cat_s,limit=2,numBuckets=4,domainSize=7}]},json={facet={cat_s={type=terms,
field=cat_s, sort=count desc, limit=2, overrequest=0,
refine=true}}},timing={time=2819.0,prepare={time=619.0,query={time=0.0},facet={time=0.0},facet_module={time=615.0},mlt={time=0.0},highlight={time=0.0},stats={time=0.0},expand={time=0.0},terms={time=0.0},debug={time=0.0}},process={time=2192.0,query={time=20.0},facet={time=0.0},facet_module={time=2095.0},mlt={time=0.0},highlight={time=0.0},stats={time=0.0},expand={time=0.0},terms={time=0.0},debug={time=74.0}}}}}"}
{code}
3rd request:
{code}
"Response":"{responseHeader={zkConnected=true,status=0,QTime=3231,params={df=_text_,distrib=false,debug=[false,
timing, track],_facet_={},fl=[id,
score],shards.purpose=1048580,start=0,fsv=true,shard.url=http://127.0.0.1:8983/solr/collection1_shard1_replica_n1/,rows=10,rid=127.0.0.1-collection1_shard3_replica_n4-1501147655592-0,version=2,q=*:*,json.facet={cat_s:{type:terms,field:cat_s,sort:\"count
desc\",limit:2,overrequest:0,refine:true}},requestPurpose=GET_TOP_IDS,NOW=1501147655590,isShard=true,wt=javabin,debugQuery=false}},response={numFound=4,start=0,maxScore=1.0,docs=[SolrDocument{id=1,
score=1.0}, SolrDocument{id=8, score=1.0}, SolrDocument{id=10, score=1.0},
SolrDocument{id=0,
score=1.0}]},sort_values={},facets={count=4,cat_s={buckets=[{val=E,count=2},
{val=A,count=1}]}},debug={facet-trace={processor=FacetQueryProcessor,elapse=1519,query=null,domainSize=4,sub-facet=[{processor=FacetFieldProcessorByArrayDV,elapse=997,field=cat_s,limit=2,numBuckets=3,domainSize=4}]},json={facet={cat_s={type=terms,
field=cat_s, sort=count desc, limit=2, overrequest=0,
refine=true}}},timing={time=2258.0,prepare={time=14.0,query={time=0.0},facet={time=0.0},facet_module={time=5.0},mlt={time=0.0},highlight={time=0.0},stats={time=0.0},expand={time=0.0},terms={time=0.0},debug={time=0.0}},process={time=2162.0,query={time=0.0},facet={time=0.0},facet_module={time=1650.0},mlt={time=1.0},highlight={time=0.0},stats={time=0.0},expand={time=0.0},terms={time=86.0},debug={time=2.0}}}}}"}}
{code}
4th request: *REFINE DEF Included* SHARD-3
{code}
"Response":"{responseHeader={zkConnected=true,status=0,QTime=34533,params={df=_text_,distrib=false,debug=[false,
timing,
track],_facet_={\"refine\":{\"cat_s\":{\"_l\":[\"A\"]}}},shards.purpose=2097152,shard.url=http://127.0.0.1:8983/solr/collection1_shard3_replica_n4/,rows=0,rid=127.0.0.1-collection1_shard3_replica_n4-1501147655592-0,version=2,q=*:*,json.facet={cat_s:{type:terms,field:cat_s,sort:\"count
desc\",limit:2,overrequest:0,refine:true}},requestPurpose=Unknown,NOW=1501147655590,isShard=true,facet=false,wt=javabin,debugQuery=false}},response={numFound=1,start=0,docs=[]},facets={cat_s={buckets=[{val=A,count=0}]}},debug={facet-trace={processor=FacetQueryProcessor,elapse=5027,query=null,domainSize=1,sub-facet=[{processor=FacetFieldProcessorByArrayDV,elapse=21,field=cat_s,limit=2,domainSize=1}]},json={facet={cat_s={type=terms,
field=cat_s, sort=count desc, limit=2, overrequest=0,
refine=true}}},timing={time=34427.0,prepare={time=3248.0,query={time=3.0},facet={time=0.0},facet_module={time=3244.0},mlt={time=0.0},highlight={time=0.0},stats={time=0.0},expand={time=0.0},terms={time=0.0},debug={time=0.0}},process={time=28945.0,query={time=0.0},facet={time=0.0},facet_module={time=5038.0},mlt={time=0.0},highlight={time=0.0},stats={time=0.0},expand={time=23903.0},terms={time=0.0},debug={time=0.0}}}}}"},
{code}
5th request:
{code}
"Response":"{responseHeader={zkConnected=true,status=0,QTime=35712,params={df=_text_,distrib=false,debug=[true,
timing,
track],shards.purpose=320,shard.url=http://127.0.0.1:8983/solr/collection1_shard2_replica_n2/,rows=10,rid=127.0.0.1-collection1_shard3_replica_n4-1501147655592-0,version=2,q=*:*,json.facet={cat_s:{type:terms,field:cat_s,sort:\"count
desc\",limit:2,overrequest:0,refine:true}},requestPurpose=GET_FIELDS,GET_DEBUG,NOW=1501147655590,ids=12,2,3,4,6,9,isShard=true,wt=javabin,debugQuery=true}},response={numFound=6,start=0,docs=[SolrDocument{id=12,
cat_s=C, add_s=E-A, _version_=1573992355678126080}, SolrDocument{id=2,
cat_s=C, add_s=D-A, _version_=1573992355673931776}, SolrDocument{id=3, cat_s=E,
add_s=C-A, _version_=1573992355674980353}, SolrDocument{id=4, cat_s=D,
add_s=B-A, _version_=1573992355673931777}, SolrDocument{id=6, cat_s=C,
add_s=E-A, _version_=1573992355676028928}, SolrDocument{id=9, cat_s=A,
add_s=C-A,
_version_=1573992355676028929}]},debug={rawquerystring=*:*,querystring=*:*,parsedquery=MatchAllDocsQuery(*:*),parsedquery_toString=*:*,explain={12=\n1.0
= *:*\n,2=\n1.0 = *:*\n,3=\n1.0 = *:*\n,4=\n1.0 = *:*\n,6=\n1.0 =
*:*\n,9=\n1.0 = *:*\n},json={facet={cat_s={type=terms, field=cat_s, sort=count
desc, limit=2, overrequest=0,
refine=true}}},QParser=LuceneQParser,timing={time=35611.0,prepare={time=10.0,query={time=3.0},facet={time=0.0},facet_module={time=3.0},mlt={time=0.0},highlight={time=0.0},stats={time=0.0},expand={time=0.0},terms={time=0.0},debug={time=0.0}},process={time=35544.0,query={time=2172.0},facet={time=0.0},facet_module={time=0.0},mlt={time=0.0},highlight={time=0.0},stats={time=0.0},expand={time=0.0},terms={time=0.0},debug={time=33370.0}}}}}"},
"http://127.0.0.1:8983/solr/collection1_shard1_replica_n1/
{code}
6th Request: *REFINE DEF Included* SHARD-1
{code}
"Response":"{responseHeader={zkConnected=true,status=0,QTime=38962,params={df=_text_,distrib=false,debug=[true,
timing,
track],_facet_={\"refine\":{\"cat_s\":{\"_l\":[\"C\"]}}},shards.purpose=2097472,shard.url=http://127.0.0.1:8983/solr/collection1_shard1_replica_n1/,rows=10,rid=127.0.0.1-collection1_shard3_replica_n4-1501147655592-0,version=2,q=*:*,json.facet={cat_s:{type:terms,field:cat_s,sort:\"count
desc\",limit:2,overrequest:0,refine:true}},requestPurpose=GET_FIELDS,GET_DEBUG,NOW=1501147655590,ids=0,1,8,10,isShard=true,wt=javabin,debugQuery=true}},response={numFound=4,start=0,docs=[SolrDocument{id=0,
cat_s=E, add_s=B-A, _version_=1573992355692806144}, SolrDocument{id=1,
cat_s=E, add_s=E-A, _version_=1573992355690708992}, SolrDocument{id=8, cat_s=A,
add_s=C-A, _version_=1573992355691757568}, SolrDocument{id=10, cat_s=D,
add_s=E-A,
_version_=1573992355691757569}]},facets={cat_s={buckets=[{val=C,count=0}]}},debug={rawquerystring=*:*,querystring=*:*,parsedquery=MatchAllDocsQuery(*:*),parsedquery_toString=*:*,explain={0=\n1.0
= *:*\n,1=\n1.0 = *:*\n,8=\n1.0 = *:*\n,10=\n1.0 =
*:*\n},facet-trace={processor=FacetQueryProcessor,elapse=35906,query=null,domainSize=4,sub-facet=[{processor=FacetFieldProcessorByArrayDV,elapse=35897,field=cat_s,limit=2,domainSize=4}]},json={facet={cat_s={type=terms,
field=cat_s, sort=count desc, limit=2, overrequest=0,
refine=true}}},QParser=LuceneQParser,timing={time=38253.0,prepare={time=2228.0,query={time=0.0},facet={time=0.0},facet_module={time=2227.0},mlt={time=0.0},highlight={time=0.0},stats={time=0.0},expand={time=0.0},terms={time=0.0},debug={time=0.0}},process={time=36015.0,query={time=97.0},facet={time=0.0},facet_module={time=35913.0},mlt={time=0.0},highlight={time=0.0},stats={time=0.0},expand={time=0.0},terms={time=0.0},debug={time=4.0}}}}}"}}
{code}
The REFINED attributes, leaves, partial or skip buckets did not get attached to
SHARD-2 in 5th request listed above. While in COUNT ASC, in almost all possible
LIMIT values, REFINED attributes are getting attached to all the SHARD
requests, (4th-6th requests).
> Facet buckets count still incorrect after passing {refine:true} | SOLR-7542
> ---------------------------------------------------------------------------
>
> Key: SOLR-11159
> URL: https://issues.apache.org/jira/browse/SOLR-11159
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Facet Module
> Reporter: Amrit Sarkar
> Attachments: COUNT_DESC_LIMIT_2, COUNT_DESC_LIMIT_3, DOCS
>
>
> I was experimenting / analysing the new *Refinement* feature in JSON Facet
> Apis introduced in SOLR-7452. Passing {{refine:true}} with the facet
> definition.
> I am listing down the test-scenarios along with test-data:
> 3 sharded collection on 3 nodes
> node/shard: bucketVal - count
> 8987: C - 1
> 8983: C - 4 D - 1 E - 1 A - 1
> 8985: E - 2 A - 1 D - 1
> Total: BUCKETS
> C - 5 E - 3 D - 2 A - 2
> It is giving accurate results for COUNT ASC, LIMIT 1 - 4
> {code}
> curl http://localhost:8983/solr/collection1/select -d
> 'q=*:*&json.facet={cat_s:{type:terms,field:cat_s,sort:"count
> asc",limit:1,overrequest:0,refine:true}}&wt=json&indent=true'
> {code}
> {code}
> "facets":{
> "count":12,
> "cat_s":{
> "buckets":[{
> "val":"A",
> "count":2}]}}}
> {code}
> {code}
> curl http://localhost:8983/solr/collection1/select -d
> 'q=*:*&json.facet={cat_s:{type:terms,field:cat_s,sort:"count
> asc",limit:2,overrequest:0,refine:true}}&wt=json&indent=true'
> {code}
> {code}
> "facets":{
> "count":12,
> "cat_s":{
> "buckets":[{
> "val":"A",
> "count":2},
> {
> "val":"D",
> "count":2}]}}}
> {code}
> *BUT, COUNT DESC, LIMIT 2 and 3*
> {code}
> curl http://localhost:8983/solr/collection1/select -d
> 'q=*:*&json.facet={cat_s:{type:terms,field:cat_s,sort:"count
> desc",limit:2,overrequest:0,refine:true}}&wt=json&indent=true'
> {code}
> {code}
> "facets":{
> "count":12,
> "cat_s":{
> "buckets":[{
> "val":"C",
> "count":5},
> {
> "val":"A",
> "count":2}]}}}
> {code}
> {code}
> curl http://localhost:8983/solr/collection1/select -d
> 'q=*:*&json.facet={cat_s:{type:terms,field:cat_s,sort:"count
> desc",limit:3,overrequest:0,refine:true}}&wt=json&indent=true'
> {code}
> {code}
> "facets":{
> "count":12,
> "cat_s":{
> "buckets":[{
> "val":"C",
> "count":5},
> {
> "val":"A",
> "count":2},
> {
> "val":"D",
> "count":2}]}}}
> {code}
> *bucketVal {{E}} and its count {{3}} is not in facet response* Pardon me if I
> am missing some configuration or this behavior is right / justified. Ideally
> we should see bucketVal E and its count 3.
> I am attaching Index DOCS, debugQuery for COUNT DESC, LIMIT 2 and LIMIT 3.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
