Re: Solr range faceting
Oh, indeed if you changed anything about the field definition and did
_not_ reindex everything from scratch your results are unreliable.
I'm overstating the case, but it'll do for now. For instance, your
facet counts should be find if you changed the stored parameter for
instance, but anything else
And it's best to index into a new collection (or core in stand-alone)
rather then re-use the current one just for paranoia's sake.
Best,
Erick
On Fri, Sep 7, 2018 at 12:18 AM Dwane Hall wrote:
>
> Thanks Erick,
>
> The field is defined as a pfloat.
>
> indexed="true" stored="true"/>
>
> I took your advice and tried smaller result range and the counts look good.
> I might try an index rebuild I’m wondering if the data has somehow been
> corrupted by a combination of old and new index mappings.
>
> Thanks again for your assistance.
>
>
> "responseHeader":{
> "zkConnected":true,
> "status":0,
> "QTime":7},
> "response":{"numFound":3,"start":0,"docs":[
> {
> "Value":34.0},
> {
> "Value":34.0},
> {
> "Value":34.0}]
> },
> "facet_counts":
> "facet_queries":{},
> "facet_ranges":{
> "Value:{
> "counts":[
> "0.0",3],
> "gap":100.0,
> "before":0,
> "after":0,
> "between":3,
> "start":0.0,
> "end":2000.0}}
>
> From: Erick Erickson
> Sent: Friday, 7 September 2018 12:54:48 PM
> To: solr-user
> Subject: Re: Solr range faceting
>
> Indeed this doesn't look right. By my count, you're missing 599 counts
> you'd expect in that range, although the after and between numbers
> total the numFound.
>
> What kind of a field is Value? Given the number of docs missing, I'd
> guess you could get the number of docs down really small and post
> them. Something like
> values 1, 2, 3, 4, 5,
> and your range query so we could try it.
>
> What is the fieldType definition and field for Value?
>
> And finally, do you get different results if you use json faceting?
>
> Best,
> Erick
> On Thu, Sep 6, 2018 at 5:51 PM Dwane Hall wrote:
> >
> > Thanks Jan that has fixed the bucket issue but I'm a little confused at why
> > zero counts exist for some buckets when they appear to be values in them?
> >
> > "response":{"numFound":869,"start":0,"docs":[
> > {
> > "Value":9475.08},
> > {
> > "Value":780.0},
> > {
> > "Value":9475.08},
> > {
> > "Value":1000.0},
> > {
> > "Value":50.0},
> > {
> > "Value":50.0},
> > {
> > "Value":0.0},
> > {
> > "Value":800.0},
> > {
> > "Value":0.0},
> > {
> > "Value":1000.0},
> > {
> > "Value":1000.0},
> > {
> > "Value":5000.0},
> > {
> > "Value":2000.0},
> > {
> >"Value":4000.0},
> > {
> > "Value":1500.0},
> > {
> > "Value":0.0},
> > {
> > "Value":1.0},
> > {
> > "Value":5000.0},
> > {
> > "Value":1000.0},
> > {
> > "Value":0.0},
> > {
> > "Value":1200.0},
> > {
> > "Value":9000.0},
> > {
> > "Value":1500.0},
> > {
> > "Value":1.0},
> > {
> > "Value":5000.0},
> > {
> > "Value":4000.0},
> > {
> > "Value":5000.0},
> > {
> > "Value":5000.0},
> > {
> > "Value":1.0},
> > {
> > "Value":1000.0}]
> > },
> >
> > "facet_counts":{
> > "facet_queries":{},
> > "facet_ranges":{
> > "Value":{
> > "counts":[
> > "0.0",9,
> >
Re: Solr range faceting
Thanks Erick,
The field is defined as a pfloat.
I took your advice and tried smaller result range and the counts look good. I
might try an index rebuild I’m wondering if the data has somehow been corrupted
by a combination of old and new index mappings.
Thanks again for your assistance.
"responseHeader":{
"zkConnected":true,
"status":0,
"QTime":7},
"response":{"numFound":3,"start":0,"docs":[
{
"Value":34.0},
{
"Value":34.0},
{
"Value":34.0}]
},
"facet_counts":
"facet_queries":{},
"facet_ranges":{
"Value:{
"counts":[
"0.0",3],
"gap":100.0,
"before":0,
"after":0,
"between":3,
"start":0.0,
"end":2000.0}}
From: Erick Erickson
Sent: Friday, 7 September 2018 12:54:48 PM
To: solr-user
Subject: Re: Solr range faceting
Indeed this doesn't look right. By my count, you're missing 599 counts
you'd expect in that range, although the after and between numbers
total the numFound.
What kind of a field is Value? Given the number of docs missing, I'd
guess you could get the number of docs down really small and post
them. Something like
values 1, 2, 3, 4, 5,
and your range query so we could try it.
What is the fieldType definition and field for Value?
And finally, do you get different results if you use json faceting?
Best,
Erick
On Thu, Sep 6, 2018 at 5:51 PM Dwane Hall wrote:
>
> Thanks Jan that has fixed the bucket issue but I'm a little confused at why
> zero counts exist for some buckets when they appear to be values in them?
>
> "response":{"numFound":869,"start":0,"docs":[
> {
> "Value":9475.08},
> {
> "Value":780.0},
> {
> "Value":9475.08},
> {
> "Value":1000.0},
> {
> "Value":50.0},
> {
> "Value":50.0},
> {
> "Value":0.0},
> {
> "Value":800.0},
> {
> "Value":0.0},
> {
> "Value":1000.0},
> {
> "Value":1000.0},
> {
> "Value":5000.0},
> {
> "Value":2000.0},
> {
>"Value":4000.0},
> {
> "Value":1500.0},
> {
> "Value":0.0},
> {
> "Value":1.0},
> {
> "Value":5000.0},
> {
> "Value":1000.0},
> {
> "Value":0.0},
> {
> "Value":1200.0},
> {
> "Value":9000.0},
> {
> "Value":1500.0},
> {
> "Value":1.0},
> {
> "Value":5000.0},
> {
> "Value":4000.0},
> {
> "Value":5000.0},
> {
> "Value":5000.0},
> {
> "Value":1.0},
> {
> "Value":1000.0}]
> },
>
> "facet_counts":{
> "facet_queries":{},
> "facet_ranges":{
> "Value":{
> "counts":[
> "0.0",9,
> "100.0",0,
> "200.0",0,
> "300.0",0,
> "400.0",80,
> "500.0",0,
> "600.0",0,
> "700.0",69,
> "800.0",0,
> "900.0",0,
> "1000.0",0,
> "1100.0",0,
> "1200.0",0,
> "1300.0",0,
> "1400.0",0,
> "1500.0",0,
> "1600.0",0,
> "1700.0",0,
> "1800.0",0,
> "1900.0",9],
> "gap":100.0,
> "before":0,
> "after":103,
> "between":766,
> "start":0.0,
> "end":2000.0}
>
> Cheers,
>
> Dwane
>
> From: Jan H?ydahl
> Sent: Friday, 7 September 2018 9:23:44 AM
> To: solr-user@lucene.apache.org
> Subject: Re: Solr range faceting
>
> Try facet.minCount=0
>
> Jan
>
> > 7. sep. 2018 kl. 01:07 skrev Dwane
Re: Solr range faceting
Indeed this doesn't look right. By my count, you're missing 599 counts
you'd expect in that range, although the after and between numbers
total the numFound.
What kind of a field is Value? Given the number of docs missing, I'd
guess you could get the number of docs down really small and post
them. Something like
values 1, 2, 3, 4, 5,
and your range query so we could try it.
What is the fieldType definition and field for Value?
And finally, do you get different results if you use json faceting?
Best,
Erick
On Thu, Sep 6, 2018 at 5:51 PM Dwane Hall wrote:
>
> Thanks Jan that has fixed the bucket issue but I'm a little confused at why
> zero counts exist for some buckets when they appear to be values in them?
>
> "response":{"numFound":869,"start":0,"docs":[
> {
> "Value":9475.08},
> {
> "Value":780.0},
> {
> "Value":9475.08},
> {
> "Value":1000.0},
> {
> "Value":50.0},
> {
> "Value":50.0},
> {
> "Value":0.0},
> {
> "Value":800.0},
> {
> "Value":0.0},
> {
> "Value":1000.0},
> {
> "Value":1000.0},
> {
> "Value":5000.0},
> {
> "Value":2000.0},
> {
>"Value":4000.0},
> {
> "Value":1500.0},
> {
> "Value":0.0},
> {
> "Value":1.0},
> {
> "Value":5000.0},
> {
> "Value":1000.0},
> {
> "Value":0.0},
> {
> "Value":1200.0},
> {
> "Value":9000.0},
> {
> "Value":1500.0},
> {
> "Value":1.0},
> {
> "Value":5000.0},
> {
> "Value":4000.0},
> {
> "Value":5000.0},
> {
> "Value":5000.0},
> {
> "Value":1.0},
> {
> "Value":1000.0}]
> },
>
> "facet_counts":{
> "facet_queries":{},
> "facet_ranges":{
> "Value":{
> "counts":[
> "0.0",9,
> "100.0",0,
> "200.0",0,
> "300.0",0,
> "400.0",80,
> "500.0",0,
> "600.0",0,
> "700.0",69,
> "800.0",0,
> "900.0",0,
> "1000.0",0,
> "1100.0",0,
> "1200.0",0,
> "1300.0",0,
> "1400.0",0,
> "1500.0",0,
> "1600.0",0,
> "1700.0",0,
> "1800.0",0,
> "1900.0",9],
> "gap":100.0,
> "before":0,
> "after":103,
> "between":766,
> "start":0.0,
> "end":2000.0}
>
> Cheers,
>
> Dwane
>
> From: Jan H?ydahl
> Sent: Friday, 7 September 2018 9:23:44 AM
> To: solr-user@lucene.apache.org
> Subject: Re: Solr range faceting
>
> Try facet.minCount=0
>
> Jan
>
> > 7. sep. 2018 kl. 01:07 skrev Dwane Hall :
> >
> > Good morning Solr community. I'm having a few facet range issues for which
> > I'd appreciate some advice when somebody gets a spare couple of minutes.
> >
> > Environment
> > Solr Cloud (7.3.1)
> > Single Shard Index, No replicas
> >
> > Facet Configuration (I'm using the request params API and useParams at
> > runtime)
> > "facet":"true",
> > "facet.mincount":1,
> > "facet.missing":"false",
> > "facet.range":"Value"
> > "f.Value.facet.range.start":0.0,
> > "f.Value.facet.range.end":2000.0,
> > "f.Value.facet.range.gap":100,
> > "f.Value.facet.range.include":"edge",
> > "f.Value.facet.range.other":"all",
> >
> > My problem
> > With my range facet configuration I'm expecting to see a facet range entry
> > for every 'step' (100 in my case) between my facet.range.start and
> > facet.range.end s
Re: Solr range faceting
Thanks Jan that has fixed the bucket issue but I'm a little confused at why
zero counts exist for some buckets when they appear to be values in them?
"response":{"numFound":869,"start":0,"docs":[
{
"Value":9475.08},
{
"Value":780.0},
{
"Value":9475.08},
{
"Value":1000.0},
{
"Value":50.0},
{
"Value":50.0},
{
"Value":0.0},
{
"Value":800.0},
{
"Value":0.0},
{
"Value":1000.0},
{
"Value":1000.0},
{
"Value":5000.0},
{
"Value":2000.0},
{
"Value":4000.0},
{
"Value":1500.0},
{
"Value":0.0},
{
"Value":1.0},
{
"Value":5000.0},
{
"Value":1000.0},
{
"Value":0.0},
{
"Value":1200.0},
{
"Value":9000.0},
{
"Value":1500.0},
{
"Value":1.0},
{
"Value":5000.0},
{
"Value":4000.0},
{
"Value":5000.0},
{
"Value":5000.0},
{
"Value":1.0},
{
"Value":1000.0}]
},
"facet_counts":{
"facet_queries":{},
"facet_ranges":{
"Value":{
"counts":[
"0.0",9,
"100.0",0,
"200.0",0,
"300.0",0,
"400.0",80,
"500.0",0,
"600.0",0,
"700.0",69,
"800.0",0,
"900.0",0,
"1000.0",0,
"1100.0",0,
"1200.0",0,
"1300.0",0,
"1400.0",0,
"1500.0",0,
"1600.0",0,
"1700.0",0,
"1800.0",0,
"1900.0",9],
"gap":100.0,
"before":0,
"after":103,
"between":766,
"start":0.0,
"end":2000.0}
Cheers,
Dwane
From: Jan H?ydahl
Sent: Friday, 7 September 2018 9:23:44 AM
To: solr-user@lucene.apache.org
Subject: Re: Solr range faceting
Try facet.minCount=0
Jan
> 7. sep. 2018 kl. 01:07 skrev Dwane Hall :
>
> Good morning Solr community. I'm having a few facet range issues for which
> I'd appreciate some advice when somebody gets a spare couple of minutes.
>
> Environment
> Solr Cloud (7.3.1)
> Single Shard Index, No replicas
>
> Facet Configuration (I'm using the request params API and useParams at
> runtime)
> "facet":"true",
> "facet.mincount":1,
> "facet.missing":"false",
> "facet.range":"Value"
> "f.Value.facet.range.start":0.0,
> "f.Value.facet.range.end":2000.0,
> "f.Value.facet.range.gap":100,
> "f.Value.facet.range.include":"edge",
> "f.Value.facet.range.other":"all",
>
> My problem
> With my range facet configuration I'm expecting to see a facet range entry
> for every 'step' (100 in my case) between my facet.range.start and
> facet.range.end settings. Something like the following 0.0,100.0,200.0,
> ...2000.0 with a sum of the number of values that occur between each range
> step. This does not appear to be the case and in some instances I don't
> appear to get counts for some range steps (800.0 and 1000.0 for example are
> present in my result set range below but I don't get a range value facets for
> these values?)
>
> Am I completely misunderstanding how range facets are supposed to work or is
> my configuration a little askew?
>
> Any advice would be greatly appreciated.
>
> The Solr Response
> "responseHeader":{
>"zkConnected":true,
>"status":0,
>"QTime":121},
>
> "response":{"numFound":869,"start":0,"docs":[
> {
>"Value":9475.08},
> {
>"Value":780.0},
> {
>"Value":1000.0},
> {
>"Value":50.0},
> {
>"Value":50.0},
> {
>"Value":0.0},
> {
>"Value":800.0},
> {
>"Value":0.0},
> {
>"Value":1000.0},
> {
>"Value":1000.0},
> {
>"Value":5000.0},
> {
>"Value":2000.0},
> {
>"Value":4000.0},
> {
>"Value":1500.0},
> {
>"Value":0.0},
> {
>"Value":1.0},
> {
>"Value":1000.0}]
> },
> "facet_counts":{
>"facet_ranges":{
> "Value":{
>"counts":[
> "0.0",9,
> "400.0",80,
> "700.0",69,
> "1900.0",9],
>"gap":100.0,
>"before":0,
>"after":103,
>"between":766,
>"start":0.0,
>"end":2000.0}}
>
> Cheers,
>
> Dwane
Re: Solr range faceting
Try facet.minCount=0
Jan
> 7. sep. 2018 kl. 01:07 skrev Dwane Hall :
>
> Good morning Solr community. I'm having a few facet range issues for which
> I'd appreciate some advice when somebody gets a spare couple of minutes.
>
> Environment
> Solr Cloud (7.3.1)
> Single Shard Index, No replicas
>
> Facet Configuration (I'm using the request params API and useParams at
> runtime)
> "facet":"true",
> "facet.mincount":1,
> "facet.missing":"false",
> "facet.range":"Value"
> "f.Value.facet.range.start":0.0,
> "f.Value.facet.range.end":2000.0,
> "f.Value.facet.range.gap":100,
> "f.Value.facet.range.include":"edge",
> "f.Value.facet.range.other":"all",
>
> My problem
> With my range facet configuration I'm expecting to see a facet range entry
> for every 'step' (100 in my case) between my facet.range.start and
> facet.range.end settings. Something like the following 0.0,100.0,200.0,
> ...2000.0 with a sum of the number of values that occur between each range
> step. This does not appear to be the case and in some instances I don't
> appear to get counts for some range steps (800.0 and 1000.0 for example are
> present in my result set range below but I don't get a range value facets for
> these values?)
>
> Am I completely misunderstanding how range facets are supposed to work or is
> my configuration a little askew?
>
> Any advice would be greatly appreciated.
>
> The Solr Response
> "responseHeader":{
>"zkConnected":true,
>"status":0,
>"QTime":121},
>
> "response":{"numFound":869,"start":0,"docs":[
> {
>"Value":9475.08},
> {
>"Value":780.0},
> {
>"Value":1000.0},
> {
>"Value":50.0},
> {
>"Value":50.0},
> {
>"Value":0.0},
> {
>"Value":800.0},
> {
>"Value":0.0},
> {
>"Value":1000.0},
> {
>"Value":1000.0},
> {
>"Value":5000.0},
> {
>"Value":2000.0},
> {
>"Value":4000.0},
> {
>"Value":1500.0},
> {
>"Value":0.0},
> {
>"Value":1.0},
> {
>"Value":1000.0}]
> },
> "facet_counts":{
>"facet_ranges":{
> "Value":{
>"counts":[
> "0.0",9,
> "400.0",80,
> "700.0",69,
> "1900.0",9],
>"gap":100.0,
>"before":0,
>"after":103,
>"between":766,
>"start":0.0,
>"end":2000.0}}
>
> Cheers,
>
> Dwane
