Re: Json Faceting Performance Issues on solr v8.7.0

[Michael Gibney]

Ah! that's significant. The latency is likely due to building the
OrdinalMap (which maps segment ords to global ords) ... "dvhash" (assuming
the relevant fields are not multivalued) will very likely work; "dvhash"
doesn't map to global ords, so doesn't need to build the OrdinalMap (which
gets built the first time it's needed per-field per-searcher).

If "dvhash" doesn't work for some reason (multivalued fields, needs to work
over broader domains, etc.?) you could probably achieve a decent result by
configuring a static warming query (newSearcher) to issue a request that
facets on the relevant fields. That will delay the opening of each new
searcher, but will ensure that user requests don't block.

SOLR-15008 _was_ actually pretty similar, with the added wrinkle of
involving distributed (multi-shard) requests (and iirc "dvhash" wouldn't
have worked in that case?)

On Fri, Feb 5, 2021 at 8:00 PM mmb1234  wrote:

> > Does this happen on a warm searcher (are subsequent requests with no
> intervening updates _ever_ fast?)?
>
> Subsequent response times very fast if searcher remains open. As a control
> test, I faceted on the same field that I used in the q param.
>
> 1. Start solr
>
> 2. Execute q=resultId:x=0
> =>  500ms
>
> 3. Execute q=resultId:x=0
> => 40,000ms
>
> 4. Execute q=resultId:x=0
> =>  150ms
>
> 5. Execute q=processId:x=0
> =>   2,500ms
>
> 6. Execute q=processId:x=0
> => 200ms
>
>
> curl
> '
> http://localhost:8983/solr/TestCollection_shard1_replica_t3/query?q=processId:-xxx-xxx-xxx-x=0
> '
> -d '
> json.facet={
> categories:{
>   "type": "terms",
>   "field" : "processId",
>   "limit" : 1
> }
> }
>
>
>
> --
> Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>

Re: Json Faceting Performance Issues on solr v8.7.0

[mmb1234]

> Does this happen on a warm searcher (are subsequent requests with no
intervening updates _ever_ fast?)?

Subsequent response times very fast if searcher remains open. As a control
test, I faceted on the same field that I used in the q param.

1. Start solr

2. Execute q=resultId:x=0
=>  500ms

3. Execute q=resultId:x=0
=> 40,000ms

4. Execute q=resultId:x=0
=>  150ms

5. Execute q=processId:x=0
=>   2,500ms

6. Execute q=processId:x=0
=> 200ms


curl
'http://localhost:8983/solr/TestCollection_shard1_replica_t3/query?q=processId:-xxx-xxx-xxx-x=0'
-d '
json.facet={
categories:{
  "type": "terms",
  "field" : "processId",
  "limit" : 1
}
}



--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Json Faceting Performance Issues on solr v8.7.0

[Michael Gibney]

Apologies, I missed deducing from the request url that you're already
talking strictly about single-shard requests (so everything I was
suggesting about shards.preference etc. is not applicable). "dvhash" is
still worth a try though, esp. with `numFound` being 943 (out of 185
million!). Does this happen on a warm searcher (are subsequent requests
with no intervening updates _ever_ fast?)?

On Fri, Feb 5, 2021 at 6:13 PM mmb1234  wrote:

> Ok. I'll try that. Meanwhile query on resultId is subsecond response. But
> the
> immediate next query for faceting takes 40+secs. The core has 185million
> docs and 63GB index size.
>
> curl
> '
> http://localhost:8983/solr/TestCollection_shard1_replica_t3/query?q=resultId:x=0
> '
> {
>   "responseHeader":{
> "zkConnected":true,
> "status":0,
> "QTime":558,
> "params":{
>   "q":"resultId:x",
>   "cache":"false",
>   "rows":"0"}},
>   "response":{"numFound":943,"start":0,"numFoundExact":true,"docs":[]
>   }}
>
>
> curl
> '
> http://localhost:8983/solr/TestCollection_shard1_replica_t3/query?q=resultId:x=0
> '
> -d '
> json.facet={
> categories:{
>   "type": "terms",
>   "field" : "resultId",
>   "limit" : 1
> }
> }'
> {
>   "responseHeader":{
> "zkConnected":true,
> "status":0,
> "QTime":43834,
> "params":{
>   "q":"resultId:x",
>   "json.facet":"{\ncategories:{\n  \"type\": \"terms\",\n
> \"field\" : \"resultId\",\n  \"limit\" : 1\n}\n}",
>   "cache":"false",
>   "rows":"0"}},
>   "response":{"numFound":943,"start":0,"numFoundExact":true,"docs":[]
>   },
>   "facets":{
> "count":943,
> "categories":{
>   "buckets":[{
>   "val":"x",
>   "count":943}]}}}
>
>
>
> --
> Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>

Re: Json Faceting Performance Issues on solr v8.7.0

[mmb1234]

Ok. I'll try that. Meanwhile query on resultId is subsecond response. But the
immediate next query for faceting takes 40+secs. The core has 185million
docs and 63GB index size.

curl
'http://localhost:8983/solr/TestCollection_shard1_replica_t3/query?q=resultId:x=0'
{
  "responseHeader":{
"zkConnected":true,
"status":0,
"QTime":558,
"params":{
  "q":"resultId:x",
  "cache":"false",
  "rows":"0"}},
  "response":{"numFound":943,"start":0,"numFoundExact":true,"docs":[]
  }}


curl
'http://localhost:8983/solr/TestCollection_shard1_replica_t3/query?q=resultId:x=0'
-d '
json.facet={
categories:{
  "type": "terms",
  "field" : "resultId",
  "limit" : 1
}
}'
{
  "responseHeader":{
"zkConnected":true,
"status":0,
"QTime":43834,
"params":{
  "q":"resultId:x",
  "json.facet":"{\ncategories:{\n  \"type\": \"terms\",\n 
\"field\" : \"resultId\",\n  \"limit\" : 1\n}\n}",
  "cache":"false",
  "rows":"0"}},
  "response":{"numFound":943,"start":0,"numFoundExact":true,"docs":[]
  },
  "facets":{
"count":943,
"categories":{
  "buckets":[{
  "val":"x",
  "count":943}]}}}



--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Json Faceting Performance Issues on solr v8.7.0

[Michael Gibney]

`resultId` sounds like it might be a relatively high-cardinality field
(lots of unique values)? What's your number of shards, and replicas per
shard? SOLR-15008 (note: not a bug) describes a situation that may be
fundamentally similar to yours (though to be sure it's impossible to say
for sure without more information):
https://issues.apache.org/jira/browse/SOLR-15008?focusedCommentId=17236213#comment-17236213

In particular, the explanation and troubleshooting advice on the linked
comment might be relevant?

"dvhash" is _not_ mentioned on that SOLR-15008, but if the `processId` main
query significantly reduces the domain -- or more specifically, if
`resultId` is high-cardinality overall, but the cardinality of `resultId`
values _associated with a particular query_ is low -- you might consider
trying `"method"="dvhash"` (which should bypass OrdinalMap creation and
array allocation, if either/both of those contribute to the latency you're
finding).

Michael

On Fri, Feb 5, 2021 at 4:42 PM mmb1234  wrote:

> Hello,
>
> I am seeing very slow response from json faceting against a single core
> (though core is shard leader in a collection).
>
> Fields processId and resultId are non-multivalued, indexed and docvalues
> string (not text).
>
> Soft Commit = 5sec (opensearcher=true) and Hard Commit = 10sec because new
> docs are constantly being indexed with 95% new and 5% overwritten
> (overwrite=true; no atomic update). Caches are not considered useful due to
> commit frequency.
>
> Solr is v8.7.0 on openjdk11.
>
> Is there any way to improve json facet QTime?
>
> ## query only
> curl
> '
> http://localhost:8983/solr/TestCollection_shard1_replica_t3/query?q=processId:-xxx-xxx-xxx-x=0
> '
> -d '
> {
>   "responseHeader":{
> "zkConnected":true,
> "status":0,
> "QTime":552,
> "params":{
>   "q":"processId:-xxx-xxx-xxx-x",
>   "cache":"false",
>   "rows":"0"}},
>   "response":{"numFound":231311,"start":0,"numFoundExact":true,"docs":[]
>   }}
>
> ## json facet takes 46secs
> curl
> '
> http://localhost:8983/solr/TestCollection_shard1_replica_t3/query?q=processId:-xxx-xxx-xxx-x=0
> '
> -d '
> json.facet={
> categories:{
>   "type": "terms",
>   "field" : "resultId",
>   "limit" : 1
> }
> }'
> {
>   "responseHeader":{
> "zkConnected":true,
> "status":0,
> "QTime":46972,
> "params":{
>   "q":"processId:-xxx-xxx-xxx-x",
>   "json.facet":"{categories:{  \"type\": \"terms\",
> \"field\" : \"resultId\",  \"limit\" : 1}}",
>   "rows":"0"}},
>   "response":{"numFound":231311,"start":0,"numFoundExact":true,"docs":[]
>   },
>   "facets":{
> "count":231311,
> "categories":{
>   "buckets":[{
>   "val":"x",
>   "count":943}]}}}
>
>
> ## visualvm CPU sampling almost all time spent in lucene:
>
> org.apache.lucene.util.PriorityQueue.downHeap() 23,009 ms
>
> org.apache.lucene.codecs.lucene80.Lucene80DocValuesProducer$TermsDict.next()
> 13,268 ms
>
>
>
> --
> Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>

Json Faceting Performance Issues on solr v8.7.0

[mmb1234]

Hello,

I am seeing very slow response from json faceting against a single core
(though core is shard leader in a collection). 

Fields processId and resultId are non-multivalued, indexed and docvalues
string (not text).

Soft Commit = 5sec (opensearcher=true) and Hard Commit = 10sec because new
docs are constantly being indexed with 95% new and 5% overwritten
(overwrite=true; no atomic update). Caches are not considered useful due to
commit frequency.

Solr is v8.7.0 on openjdk11.

Is there any way to improve json facet QTime?

## query only
curl
'http://localhost:8983/solr/TestCollection_shard1_replica_t3/query?q=processId:-xxx-xxx-xxx-x=0'
-d '
{
  "responseHeader":{
"zkConnected":true,
"status":0,
"QTime":552,
"params":{
  "q":"processId:-xxx-xxx-xxx-x",
  "cache":"false",
  "rows":"0"}},
  "response":{"numFound":231311,"start":0,"numFoundExact":true,"docs":[]
  }}

## json facet takes 46secs
curl
'http://localhost:8983/solr/TestCollection_shard1_replica_t3/query?q=processId:-xxx-xxx-xxx-x=0'
-d '
json.facet={
categories:{
  "type": "terms",
  "field" : "resultId",
  "limit" : 1
}
}'
{
  "responseHeader":{
"zkConnected":true,
"status":0,
"QTime":46972,
"params":{
  "q":"processId:-xxx-xxx-xxx-x",
  "json.facet":"{categories:{  \"type\": \"terms\", 
\"field\" : \"resultId\",  \"limit\" : 1}}",
  "rows":"0"}},
  "response":{"numFound":231311,"start":0,"numFoundExact":true,"docs":[]
  },
  "facets":{
"count":231311,
"categories":{
  "buckets":[{
  "val":"x",
  "count":943}]}}}


## visualvm CPU sampling almost all time spent in lucene:

org.apache.lucene.util.PriorityQueue.downHeap() 23,009 ms
org.apache.lucene.codecs.lucene80.Lucene80DocValuesProducer$TermsDict.next()
13,268 ms



--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Multi-select faceting for nested documents

[Lance Snell]

-1
childFilter='/productItems/{!filters v=$child_FQ2}']"
,
"filter": [

// "${prdoc_Q2} {!parent which='*:* -_nest_path_:/*' v=$prdoc_Q2}
{!parent which='*:* -_nest_path_:/productItems/*' v=$prdoc_Q2}
categoryPrefixes:(ASB00434166111481 ASB0043416611148All)"

"_query_:(${prdoc_Q2} ${pcdoc_Q2} docType:pcdoc docType:prdoc
docType:pidoc docType:pdoc)"
,
"{!filters tag=MID param=$test v=$baseQ}"
],
"sort":
"{!parent which='*:* -_nest_path_:*' score=max v='+docType:prdoc
+{!func}fprice'} asc"
,
"offset": 0,
"limit": "${LIMIT}",
"facet": {
"testing": {
"type": "terms",
"field": "docType",
"limit": -1,
"facet": {
"parentCount": "unique(_root_)",
},
"domain":{
"excludeTags":[
"TOP",
//"MID",
// "LOW"
]
}
},
"testing2": {
"type": "terms",
"field": "docType",
"limit": -1,
"facet": {
"parentCount": "unique(_root_)",
},
"domain":{
"excludeTags":[
"TOP",
"MID",
// "LOW"
    ]
}
}
}
}


On Mon, Jan 25, 2021 at 9:41 AM Alexandre Rafalovitch 
wrote:

> I don't have an answer, but I feel that maybe explaining the situation
> in more details would help a bit more. Specifically, you explain your
> data structure well, but not your actual presentation requirement in
> enough details.
>
> How would you like the multi-select to work, how it is working for you
> now and what is the gap?
>
> Regards,
>Alex.
> P.s. Sometimes, you have to really modify the way the information is
> stored in Solr for the efficient and effective search results. Solr is
> not the database, so it needs to model search requirements, rather
> than original data shape.
>
> On Mon, 25 Jan 2021 at 10:34, Lance Snell 
> wrote:
> >
> > Any examples would be greatly appreciated.
> >
> > On Mon, Jan 25, 2021, 2:25 AM Lance Snell 
> wrote:
> >
> > > Hey all,
> > >
> > > I am having trouble finding current examples of multi-select faceting
> for
> > > nested documents.  Specifically ones with *multiple *levels of nested
> > > documents.
> > >
> > > My current schema has a parent document, two child documents(siblings),
> > > and a grandchild document.  I am using the JSON API.
> > >
> > > Product -> Sku -> Price
> > >|
> > >\/
> > > StoreCategory
> > >
> > > Any help/direction would be appreciated.
> > >
> > >
> > > Solr. 8.6
> > >
> > > --
> > > Thanks,
> > >
> > > Lance
> > >
>


-- 
Thanks,

Lance Snell
(507) 829-7389

Re: Multi-select faceting for nested documents

[Alexandre Rafalovitch]

I don't have an answer, but I feel that maybe explaining the situation
in more details would help a bit more. Specifically, you explain your
data structure well, but not your actual presentation requirement in
enough details.

How would you like the multi-select to work, how it is working for you
now and what is the gap?

Regards,
   Alex.
P.s. Sometimes, you have to really modify the way the information is
stored in Solr for the efficient and effective search results. Solr is
not the database, so it needs to model search requirements, rather
than original data shape.

On Mon, 25 Jan 2021 at 10:34, Lance Snell  wrote:
>
> Any examples would be greatly appreciated.
>
> On Mon, Jan 25, 2021, 2:25 AM Lance Snell  wrote:
>
> > Hey all,
> >
> > I am having trouble finding current examples of multi-select faceting for
> > nested documents.  Specifically ones with *multiple *levels of nested
> > documents.
> >
> > My current schema has a parent document, two child documents(siblings),
> > and a grandchild document.  I am using the JSON API.
> >
> > Product -> Sku -> Price
> >|
> >\/
> > StoreCategory
> >
> > Any help/direction would be appreciated.
> >
> >
> > Solr. 8.6
> >
> > --
> > Thanks,
> >
> > Lance
> >

Re: Multi-select faceting for nested documents

[Lance Snell]

Any examples would be greatly appreciated.

On Mon, Jan 25, 2021, 2:25 AM Lance Snell  wrote:

> Hey all,
>
> I am having trouble finding current examples of multi-select faceting for
> nested documents.  Specifically ones with *multiple *levels of nested
> documents.
>
> My current schema has a parent document, two child documents(siblings),
> and a grandchild document.  I am using the JSON API.
>
> Product -> Sku -> Price
>|
>\/
> StoreCategory
>
> Any help/direction would be appreciated.
>
>
> Solr. 8.6
>
> --
> Thanks,
>
> Lance
>

Multi-select faceting for nested documents

[Lance Snell]

Hey all,

I am having trouble finding current examples of multi-select faceting for
nested documents.  Specifically ones with *multiple *levels of nested
documents.

My current schema has a parent document, two child documents(siblings), and
a grandchild document.  I am using the JSON API.

Product -> Sku -> Price
   |
   \/
StoreCategory

Any help/direction would be appreciated.


Solr. 8.6

-- 
Thanks,

Lance

Re: Range faceting on timestamp field

[Erick Erickson]

Then you need to form your start at relative to your timezone.

What I’d actually recommend is that if you need to bucket by day,
you index the day in a separate field. Of course, if you have to
bucket by day in arbitrary timezones that won’t work…..

Best,
Erick

> On Dec 24, 2020, at 4:42 PM, ufuk yılmaz  wrote:
> 
> Hello all,
> 
> I have a plong field in my schema representing a Unix timestamp
> 
> 
> 
> I’m doing a range facet over this field to find which event occured on which 
> day. I’m setting “start” on some date at 00:00 o’clock, end on another, and 
> setting gap to 86400 (total seconds in a day)
> ...
> "type": "range",
> "field": "timestamp_s",
> "start": 1338498000,
> "end": 1339275600,
> "gap": 86400,
> ...
> 
> Lets say that an event occured at 19:00 GMT+00. This facet puts it in the 
> bucket of that day, which starts at 00:00. I’m living in GMT+2 timezone, so 
> clock was 21:00 and that event occured on the same day with me, which is all 
> good and correct.
> 
> Another event occured at 23:00 GMT+00, Day 2. At that time, it was 01:00 Day 
> 3 here. Faceting puts the event at Day 2 00:00’s bucket, when converted to my 
> timezone, puts the event on Day 2. But it was Day 3 here when the event 
> happened...
> 
> I wish I didn’t bore the hell out of you. Do you have any suggestion to solve 
> this problem? Unfortunately my timestamp field is not a date field and I need 
> to show the results from my perspective, not from the universal time.
> 
> Have a nice day!
> 
> Sent from Mail for Windows 10
> 

Range faceting on timestamp field

[ufuk yılmaz]

Hello all,

I have a plong field in my schema representing a Unix timestamp



I’m doing a range facet over this field to find which event occured on which 
day. I’m setting “start” on some date at 00:00 o’clock, end on another, and 
setting gap to 86400 (total seconds in a day)
...
"type": "range",
"field": "timestamp_s",
"start": 1338498000,
"end": 1339275600,
"gap": 86400,
...

Lets say that an event occured at 19:00 GMT+00. This facet puts it in the 
bucket of that day, which starts at 00:00. I’m living in GMT+2 timezone, so 
clock was 21:00 and that event occured on the same day with me, which is all 
good and correct.

Another event occured at 23:00 GMT+00, Day 2. At that time, it was 01:00 Day 3 
here. Faceting puts the event at Day 2 00:00’s bucket, when converted to my 
timezone, puts the event on Day 2. But it was Day 3 here when the event 
happened...

I wish I didn’t bore the hell out of you. Do you have any suggestion to solve 
this problem? Unfortunately my timestamp field is not a date field and I need 
to show the results from my perspective, not from the universal time.

Have a nice day!

Sent from Mail for Windows 10

Re: increasing number of threads for faceting in JSON format

[Arturas Mazeika]

Hi Christine,

Thanks a lot for the posts. Very impressive information (article as well as
the youtube video!)

Thanks a lot Merry Xmas and and Happy New Year!

Cheers,
Arturas

On Thu, Dec 24, 2020 at 11:03 AM Christine Poerschke (BLOOMBERG/ LONDON) <
cpoersc...@bloomberg.net> wrote:

> Hello again Arturas.
>
> I meant to reply before but somehow lost track of it ... The "Lifecycle of
> a Solr Search Request" slides [1] and/or talk [2] may be of interest to you.
>
> Regards,
> Christine
>
> [1] https://home.apache.org/~hossman/rev2017/
> [2] https://youtu.be/qItRilJLj5o
>
> From: solr-user@lucene.apache.org At: 12/10/20 21:42:19To:
> solr-user@lucene.apache.org
> Subject: Re: increasing number of threads for faceting in JSON format
>
> Hi Christine Munendra et al,
>
> Wow, you dag into the code and checked weather threads are being blown in
> range and term queries! I wish one day to be able to do the same myself.
>
> How does one get to the level, so one can check the code herself? Is there
> like a nice primer or crash course, solr 101 so to say, things you did not
> learn in school about solr, but you wish you had learned web page? Well,
> I'll take this opportunity to scroll through the lines in the github. Your
> answer is very helpful.
>
> Cheers,
> Arturas
>
> On Thu, Dec 10, 2020 at 7:08 PM Munendra S N 
> wrote:
>
> > Thank you Christine.
> > Yeah, JSON facet does not support specifying threads.
> >
> >
> > On Thu, Dec 10, 2020, 11:15 PM Christine Poerschke (BLOOMBERG/ LONDON) <
> > cpoersc...@bloomberg.net> wrote:
> >
> > > Hello Arturas and Munendra!
> > >
> > > In the "Currently, JSON facets have support for specifying the number
> of
> > > threads." sentence, I wonder if perhaps a "does not" got inadvertently
> > > omitted i.e. "Currently, JSON facets does not have support for
> specifying
> > > the number of threads." was intended?
> > >
> > > Let me share what I learnt from digging into the code:
> > >
> > > * "facet.threads" is for field value faceting [1] [2] but you're
> > > interested in (JSON) field range faceting as well as JSON field value
> > > faceting.
> > >
> > > * The area of the code [3] that does the JSON field range faceting
> shows
> > > no obvious threading or parallelisation.
> > >
> > > Hope that helps?
> > >
> > > Regards,
> > >
> > > Christine
> > >
> > > [1]
> > >
> >
>
> https://lucene.apache.org/solr/guide/8_7/faceting.html#field-value-faceting-para
> meters
> > > [2]
> > >
> >
>
> https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.7.0/solr/core/
> src/java/org/apache/solr/request/SimpleFacets.java
> <https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.7.0/solr/core/src/java/org/apache/solr/request/SimpleFacets.java>
> > > [3]
> > >
> >
>
> https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.7.0/solr/core/
> src/java/org/apache/solr/search/facet/FacetRangeProcessor.java#L112-L113
> <https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.7.0/solr/core/src/java/org/apache/solr/search/facet/FacetRangeProcessor.java#L112-L113>
> > >
> > > From: solr-user@lucene.apache.org At: 12/03/20 22:47:35To:
> > > solr-user@lucene.apache.org
> > > Subject: Re: increasing number of threads for faceting in JSON format
> > >
> > > Hi Munedra,
> > >
> > > This is great that I can get things faster by reducing the gap and by
> > > increasing the number of threads. How to reduce gaps I know: one can
> > > replace   "gap":   "+1HOUR" with   "gap":   "+1MONTH" What should I
> > change
> > > in the text below to increase the number of threads from one to 20?
> > >
> > > Cheers,
> > > Arturas
> > >
> > > On Thu, Dec 3, 2020 at 1:54 PM Munendra S N 
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > Currently, JSON facets have support for specifying the number of
> > threads.
> > > > In the above request, the range facet is computed over 2 years with a
> > gap
> > > > of 1 hour. By reducing the number of buckets, computation should
> become
> > > > much faster
> > > >
> > > > Regards,
> > > > Munendra S N
> > > >
> > > >
> > > >
> > > &g

Re: increasing number of threads for faceting in JSON format

[Christine Poerschke (BLOOMBERG/ LONDON)]

Hello again Arturas.

I meant to reply before but somehow lost track of it ... The "Lifecycle of a 
Solr Search Request" slides [1] and/or talk [2] may be of interest to you.

Regards,
Christine

[1] https://home.apache.org/~hossman/rev2017/
[2] https://youtu.be/qItRilJLj5o

From: solr-user@lucene.apache.org At: 12/10/20 21:42:19To:  
solr-user@lucene.apache.org
Subject: Re: increasing number of threads for faceting in JSON format

Hi Christine Munendra et al,

Wow, you dag into the code and checked weather threads are being blown in
range and term queries! I wish one day to be able to do the same myself.

How does one get to the level, so one can check the code herself? Is there
like a nice primer or crash course, solr 101 so to say, things you did not
learn in school about solr, but you wish you had learned web page? Well,
I'll take this opportunity to scroll through the lines in the github. Your
answer is very helpful.

Cheers,
Arturas

On Thu, Dec 10, 2020 at 7:08 PM Munendra S N 
wrote:

> Thank you Christine.
> Yeah, JSON facet does not support specifying threads.
>
>
> On Thu, Dec 10, 2020, 11:15 PM Christine Poerschke (BLOOMBERG/ LONDON) <
> cpoersc...@bloomberg.net> wrote:
>
> > Hello Arturas and Munendra!
> >
> > In the "Currently, JSON facets have support for specifying the number of
> > threads." sentence, I wonder if perhaps a "does not" got inadvertently
> > omitted i.e. "Currently, JSON facets does not have support for specifying
> > the number of threads." was intended?
> >
> > Let me share what I learnt from digging into the code:
> >
> > * "facet.threads" is for field value faceting [1] [2] but you're
> > interested in (JSON) field range faceting as well as JSON field value
> > faceting.
> >
> > * The area of the code [3] that does the JSON field range faceting shows
> > no obvious threading or parallelisation.
> >
> > Hope that helps?
> >
> > Regards,
> >
> > Christine
> >
> > [1]
> >
> 
https://lucene.apache.org/solr/guide/8_7/faceting.html#field-value-faceting-para
meters
> > [2]
> >
> 
https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.7.0/solr/core/
src/java/org/apache/solr/request/SimpleFacets.java
> > [3]
> >
> 
https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.7.0/solr/core/
src/java/org/apache/solr/search/facet/FacetRangeProcessor.java#L112-L113
> >
> > From: solr-user@lucene.apache.org At: 12/03/20 22:47:35To:
> > solr-user@lucene.apache.org
> > Subject: Re: increasing number of threads for faceting in JSON format
> >
> > Hi Munedra,
> >
> > This is great that I can get things faster by reducing the gap and by
> > increasing the number of threads. How to reduce gaps I know: one can
> > replace   "gap":   "+1HOUR" with   "gap":   "+1MONTH" What should I
> change
> > in the text below to increase the number of threads from one to 20?
> >
> > Cheers,
> > Arturas
> >
> > On Thu, Dec 3, 2020 at 1:54 PM Munendra S N 
> > wrote:
> >
> > > Hi,
> > >
> > > Currently, JSON facets have support for specifying the number of
> threads.
> > > In the above request, the range facet is computed over 2 years with a
> gap
> > > of 1 hour. By reducing the number of buckets, computation should become
> > > much faster
> > >
> > > Regards,
> > > Munendra S N
> > >
> > >
> > >
> > > On Thu, Dec 3, 2020 at 1:52 PM Arturas Mazeika 
> > wrote:
> > >
> > > > Hi Solr-Users,
> > > >
> > > > I am trying to better understand the solr capabilities, how one can
> > > > formulate queries in JSON format as well as tweak parameters.
> > Currently I
> > > > have a logs collection (ca 6GB large) with a dozen of attributes
> > running
> > > in
> > > > single server mode (F:\solr_deployment\solr-8.7.0\bin\solr.cmd start
> -h
> > > > localhost -p  -m 4g)
> > > >
> > > > I am playing with faceting functionality in solr and query a couple
> of
> > > > attributes there. My typical query is:
> > > >
> > > > GET http://localhost:/solr/db/query
> > > > <http://arteika:/solr/logan/query> HTTP/1.1
> > > > content-type: application/json
> > > >
> > > > {
> > > > "query"  : "*:*",
> > > > "limit"  : 0,
> > > > "facet": 

Re: increasing number of threads for faceting in JSON format

[Arturas Mazeika]

Hi Christine Munendra et al,

Wow, you dag into the code and checked weather threads are being blown in
range and term queries! I wish one day to be able to do the same myself.

How does one get to the level, so one can check the code herself? Is there
like a nice primer or crash course, solr 101 so to say, things you did not
learn in school about solr, but you wish you had learned web page? Well,
I'll take this opportunity to scroll through the lines in the github. Your
answer is very helpful.

Cheers,
Arturas

On Thu, Dec 10, 2020 at 7:08 PM Munendra S N 
wrote:

> Thank you Christine.
> Yeah, JSON facet does not support specifying threads.
>
>
> On Thu, Dec 10, 2020, 11:15 PM Christine Poerschke (BLOOMBERG/ LONDON) <
> cpoersc...@bloomberg.net> wrote:
>
> > Hello Arturas and Munendra!
> >
> > In the "Currently, JSON facets have support for specifying the number of
> > threads." sentence, I wonder if perhaps a "does not" got inadvertently
> > omitted i.e. "Currently, JSON facets does not have support for specifying
> > the number of threads." was intended?
> >
> > Let me share what I learnt from digging into the code:
> >
> > * "facet.threads" is for field value faceting [1] [2] but you're
> > interested in (JSON) field range faceting as well as JSON field value
> > faceting.
> >
> > * The area of the code [3] that does the JSON field range faceting shows
> > no obvious threading or parallelisation.
> >
> > Hope that helps?
> >
> > Regards,
> >
> > Christine
> >
> > [1]
> >
> https://lucene.apache.org/solr/guide/8_7/faceting.html#field-value-faceting-parameters
> > [2]
> >
> https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.7.0/solr/core/src/java/org/apache/solr/request/SimpleFacets.java
> > [3]
> >
> https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.7.0/solr/core/src/java/org/apache/solr/search/facet/FacetRangeProcessor.java#L112-L113
> >
> > From: solr-user@lucene.apache.org At: 12/03/20 22:47:35To:
> > solr-user@lucene.apache.org
> > Subject: Re: increasing number of threads for faceting in JSON format
> >
> > Hi Munedra,
> >
> > This is great that I can get things faster by reducing the gap and by
> > increasing the number of threads. How to reduce gaps I know: one can
> > replace   "gap":   "+1HOUR" with   "gap":   "+1MONTH" What should I
> change
> > in the text below to increase the number of threads from one to 20?
> >
> > Cheers,
> > Arturas
> >
> > On Thu, Dec 3, 2020 at 1:54 PM Munendra S N 
> > wrote:
> >
> > > Hi,
> > >
> > > Currently, JSON facets have support for specifying the number of
> threads.
> > > In the above request, the range facet is computed over 2 years with a
> gap
> > > of 1 hour. By reducing the number of buckets, computation should become
> > > much faster
> > >
> > > Regards,
> > > Munendra S N
> > >
> > >
> > >
> > > On Thu, Dec 3, 2020 at 1:52 PM Arturas Mazeika 
> > wrote:
> > >
> > > > Hi Solr-Users,
> > > >
> > > > I am trying to better understand the solr capabilities, how one can
> > > > formulate queries in JSON format as well as tweak parameters.
> > Currently I
> > > > have a logs collection (ca 6GB large) with a dozen of attributes
> > running
> > > in
> > > > single server mode (F:\solr_deployment\solr-8.7.0\bin\solr.cmd start
> -h
> > > > localhost -p  -m 4g)
> > > >
> > > > I am playing with faceting functionality in solr and query a couple
> of
> > > > attributes there. My typical query is:
> > > >
> > > > GET http://localhost:/solr/db/query
> > > > <http://arteika:/solr/logan/query> HTTP/1.1
> > > > content-type: application/json
> > > >
> > > > {
> > > > "query"  : "*:*",
> > > > "limit"  : 0,
> > > > "facet": {
> > > > "t" : {
> > > > "type":  "terms",
> > > > "field": "fcomp",
> > > > "sort":  "index",
> > > >
> > > > "facet": {
> > > > "t_buckets": {
> > > > "type":  "range",
> >

Re: increasing number of threads for faceting in JSON format

[Munendra S N]

Thank you Christine.
Yeah, JSON facet does not support specifying threads.


On Thu, Dec 10, 2020, 11:15 PM Christine Poerschke (BLOOMBERG/ LONDON) <
cpoersc...@bloomberg.net> wrote:

> Hello Arturas and Munendra!
>
> In the "Currently, JSON facets have support for specifying the number of
> threads." sentence, I wonder if perhaps a "does not" got inadvertently
> omitted i.e. "Currently, JSON facets does not have support for specifying
> the number of threads." was intended?
>
> Let me share what I learnt from digging into the code:
>
> * "facet.threads" is for field value faceting [1] [2] but you're
> interested in (JSON) field range faceting as well as JSON field value
> faceting.
>
> * The area of the code [3] that does the JSON field range faceting shows
> no obvious threading or parallelisation.
>
> Hope that helps?
>
> Regards,
>
> Christine
>
> [1]
> https://lucene.apache.org/solr/guide/8_7/faceting.html#field-value-faceting-parameters
> [2]
> https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.7.0/solr/core/src/java/org/apache/solr/request/SimpleFacets.java
> [3]
> https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.7.0/solr/core/src/java/org/apache/solr/search/facet/FacetRangeProcessor.java#L112-L113
>
> From: solr-user@lucene.apache.org At: 12/03/20 22:47:35To:
> solr-user@lucene.apache.org
> Subject: Re: increasing number of threads for faceting in JSON format
>
> Hi Munedra,
>
> This is great that I can get things faster by reducing the gap and by
> increasing the number of threads. How to reduce gaps I know: one can
> replace   "gap":   "+1HOUR" with   "gap":   "+1MONTH" What should I change
> in the text below to increase the number of threads from one to 20?
>
> Cheers,
> Arturas
>
> On Thu, Dec 3, 2020 at 1:54 PM Munendra S N 
> wrote:
>
> > Hi,
> >
> > Currently, JSON facets have support for specifying the number of threads.
> > In the above request, the range facet is computed over 2 years with a gap
> > of 1 hour. By reducing the number of buckets, computation should become
> > much faster
> >
> > Regards,
> > Munendra S N
> >
> >
> >
> > On Thu, Dec 3, 2020 at 1:52 PM Arturas Mazeika 
> wrote:
> >
> > > Hi Solr-Users,
> > >
> > > I am trying to better understand the solr capabilities, how one can
> > > formulate queries in JSON format as well as tweak parameters.
> Currently I
> > > have a logs collection (ca 6GB large) with a dozen of attributes
> running
> > in
> > > single server mode (F:\solr_deployment\solr-8.7.0\bin\solr.cmd start -h
> > > localhost -p  -m 4g)
> > >
> > > I am playing with faceting functionality in solr and query a couple of
> > > attributes there. My typical query is:
> > >
> > > GET http://localhost:/solr/db/query
> > > <http://arteika:/solr/logan/query> HTTP/1.1
> > > content-type: application/json
> > >
> > > {
> > > "query"  : "*:*",
> > > "limit"  : 0,
> > > "facet": {
> > > "t" : {
> > > "type":  "terms",
> > > "field": "fcomp",
> > > "sort":  "index",
> > >
> > > "facet": {
> > > "t_buckets": {
> > > "type":  "range",
> > > "field": "t",
> > > "sort": { "t": "asc" },
> > > "start": "2018-05-02T17:00:00.000Z",
> > > "end":   "2020-11-16T21:00:00.000Z",
> > > "gap":   "+1HOUR"
> > > }
> > > }
> > > },
> > > }
> > > }
> > >
> > > not surprisingly, it takes a bit to compute the result, so I tried to
> > > increase the number of threads. How do I do it in JSON format? I tried
> > > adding
> > >
> > > {
> > > "params": {
> > > "facet.threads": 8
> > > },
> > > "query"  : "*:*",
> > > ...
> > > }
> > >
> > > and checked the jstack  of the solr java process, but I still see
> > only
> > > one thread working.  Can I configure params through the params section?
> > >
> > > I also tried
> > >
> > > {
> > > "query"  : "*:*",
> > > "limit"  : 0,
> > > "facet": {
> > > "t" : {
> > > "type":  "terms",
> > > "field": "fcomp",
> > > "sort":  "index",
> > >
> > > "facet": {
> > > "t_buckets": {
> > > "type":  "range",
> > > "field": "t",
> > > "sort": { "t": "asc" },
> > > "start": "2018-05-02T17:00:00.000Z",
> > > "end":   "2020-11-16T21:00:00.000Z",
> > > "gap":   "+1HOUR"
> > > }
> > > },
> > > "threads":8
> > > },
> > > }
> > > }
> > >
> > > but this ran in one thread as well. Can I influence the number of
> threads
> > > in the "facet" section of JSON?
> > >
> > > Cheers,
> > > Arturas
> > >
> >
>
>
>

Re: increasing number of threads for faceting in JSON format

[Christine Poerschke (BLOOMBERG/ LONDON)]

Hello Arturas and Munendra!

In the "Currently, JSON facets have support for specifying the number of 
threads." sentence, I wonder if perhaps a "does not" got inadvertently omitted 
i.e. "Currently, JSON facets does not have support for specifying the number of 
threads." was intended?

Let me share what I learnt from digging into the code:

* "facet.threads" is for field value faceting [1] [2] but you're interested in 
(JSON) field range faceting as well as JSON field value faceting.

* The area of the code [3] that does the JSON field range faceting shows no 
obvious threading or parallelisation.

Hope that helps?

Regards,

Christine

[1] 
https://lucene.apache.org/solr/guide/8_7/faceting.html#field-value-faceting-parameters
[2] 
https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.7.0/solr/core/src/java/org/apache/solr/request/SimpleFacets.java
[3] 
https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.7.0/solr/core/src/java/org/apache/solr/search/facet/FacetRangeProcessor.java#L112-L113

From: solr-user@lucene.apache.org At: 12/03/20 22:47:35To:  
solr-user@lucene.apache.org
Subject: Re: increasing number of threads for faceting in JSON format

Hi Munedra,

This is great that I can get things faster by reducing the gap and by
increasing the number of threads. How to reduce gaps I know: one can
replace   "gap":   "+1HOUR" with   "gap":   "+1MONTH" What should I change
in the text below to increase the number of threads from one to 20?

Cheers,
Arturas

On Thu, Dec 3, 2020 at 1:54 PM Munendra S N  wrote:

> Hi,
>
> Currently, JSON facets have support for specifying the number of threads.
> In the above request, the range facet is computed over 2 years with a gap
> of 1 hour. By reducing the number of buckets, computation should become
> much faster
>
> Regards,
> Munendra S N
>
>
>
> On Thu, Dec 3, 2020 at 1:52 PM Arturas Mazeika  wrote:
>
> > Hi Solr-Users,
> >
> > I am trying to better understand the solr capabilities, how one can
> > formulate queries in JSON format as well as tweak parameters. Currently I
> > have a logs collection (ca 6GB large) with a dozen of attributes running
> in
> > single server mode (F:\solr_deployment\solr-8.7.0\bin\solr.cmd start -h
> > localhost -p  -m 4g)
> >
> > I am playing with faceting functionality in solr and query a couple of
> > attributes there. My typical query is:
> >
> > GET http://localhost:/solr/db/query
> > <http://arteika:/solr/logan/query> HTTP/1.1
> > content-type: application/json
> >
> > {
> > "query"  : "*:*",
> > "limit"  : 0,
> > "facet": {
> > "t" : {
> > "type":  "terms",
> > "field": "fcomp",
> > "sort":  "index",
> >
> > "facet": {
> > "t_buckets": {
> > "type":  "range",
> > "field": "t",
> > "sort": { "t": "asc" },
> > "start": "2018-05-02T17:00:00.000Z",
> > "end":   "2020-11-16T21:00:00.000Z",
> > "gap":   "+1HOUR"
> > }
> > }
> > },
> > }
> > }
> >
> > not surprisingly, it takes a bit to compute the result, so I tried to
> > increase the number of threads. How do I do it in JSON format? I tried
> > adding
> >
> > {
> > "params": {
> > "facet.threads": 8
> > },
> > "query"  : "*:*",
> > ...
> > }
> >
> > and checked the jstack  of the solr java process, but I still see
> only
> > one thread working.  Can I configure params through the params section?
> >
> > I also tried
> >
> > {
> > "query"  : "*:*",
> > "limit"  : 0,
> > "facet": {
> > "t" : {
> > "type":  "terms",
> > "field": "fcomp",
> > "sort":  "index",
> >
> > "facet": {
> > "t_buckets": {
> > "type":  "range",
> > "field": "t",
> > "sort": { "t": "asc" },
> > "start": "2018-05-02T17:00:00.000Z",
> > "end":   "2020-11-16T21:00:00.000Z",
> > "gap":   "+1HOUR"
> > }
> > },
> > "threads":8
> > },
> > }
> > }
> >
> > but this ran in one thread as well. Can I influence the number of threads
> > in the "facet" section of JSON?
> >
> > Cheers,
> > Arturas
> >
>

Re: increasing number of threads for faceting in JSON format

[Arturas Mazeika]

Hi Munedra,

This is great that I can get things faster by reducing the gap and by
increasing the number of threads. How to reduce gaps I know: one can
replace   "gap":   "+1HOUR" with   "gap":   "+1MONTH" What should I change
in the text below to increase the number of threads from one to 20?

Cheers,
Arturas

On Thu, Dec 3, 2020 at 1:54 PM Munendra S N  wrote:

> Hi,
>
> Currently, JSON facets have support for specifying the number of threads.
> In the above request, the range facet is computed over 2 years with a gap
> of 1 hour. By reducing the number of buckets, computation should become
> much faster
>
> Regards,
> Munendra S N
>
>
>
> On Thu, Dec 3, 2020 at 1:52 PM Arturas Mazeika  wrote:
>
> > Hi Solr-Users,
> >
> > I am trying to better understand the solr capabilities, how one can
> > formulate queries in JSON format as well as tweak parameters. Currently I
> > have a logs collection (ca 6GB large) with a dozen of attributes running
> in
> > single server mode (F:\solr_deployment\solr-8.7.0\bin\solr.cmd start -h
> > localhost -p  -m 4g)
> >
> > I am playing with faceting functionality in solr and query a couple of
> > attributes there. My typical query is:
> >
> > GET http://localhost:/solr/db/query
> > <http://arteika:/solr/logan/query> HTTP/1.1
> > content-type: application/json
> >
> > {
> > "query"  : "*:*",
> > "limit"  : 0,
> > "facet": {
> > "t" : {
> > "type":  "terms",
> > "field": "fcomp",
> > "sort":  "index",
> >
> > "facet": {
> > "t_buckets": {
> > "type":  "range",
> > "field": "t",
> > "sort": { "t": "asc" },
> > "start": "2018-05-02T17:00:00.000Z",
> > "end":   "2020-11-16T21:00:00.000Z",
> > "gap":   "+1HOUR"
> > }
> > }
> > },
> > }
> > }
> >
> > not surprisingly, it takes a bit to compute the result, so I tried to
> > increase the number of threads. How do I do it in JSON format? I tried
> > adding
> >
> > {
> > "params": {
> > "facet.threads": 8
> > },
> > "query"  : "*:*",
> > ...
> > }
> >
> > and checked the jstack  of the solr java process, but I still see
> only
> > one thread working.  Can I configure params through the params section?
> >
> > I also tried
> >
> > {
> > "query"  : "*:*",
> > "limit"  : 0,
> > "facet": {
> > "t" : {
> > "type":  "terms",
> > "field": "fcomp",
> > "sort":  "index",
> >
> > "facet": {
> > "t_buckets": {
> > "type":  "range",
> > "field": "t",
> > "sort": { "t": "asc" },
> > "start": "2018-05-02T17:00:00.000Z",
> > "end":   "2020-11-16T21:00:00.000Z",
> > "gap":   "+1HOUR"
> > }
> > },
> > "threads":8
> > },
> > }
> > }
> >
> > but this ran in one thread as well. Can I influence the number of threads
> > in the "facet" section of JSON?
> >
> > Cheers,
> > Arturas
> >
>

Re: increasing number of threads for faceting in JSON format

[Munendra S N]

Hi,

Currently, JSON facets have support for specifying the number of threads.
In the above request, the range facet is computed over 2 years with a gap
of 1 hour. By reducing the number of buckets, computation should become
much faster

Regards,
Munendra S N



On Thu, Dec 3, 2020 at 1:52 PM Arturas Mazeika  wrote:

> Hi Solr-Users,
>
> I am trying to better understand the solr capabilities, how one can
> formulate queries in JSON format as well as tweak parameters. Currently I
> have a logs collection (ca 6GB large) with a dozen of attributes running in
> single server mode (F:\solr_deployment\solr-8.7.0\bin\solr.cmd start -h
> localhost -p  -m 4g)
>
> I am playing with faceting functionality in solr and query a couple of
> attributes there. My typical query is:
>
> GET http://localhost:/solr/db/query
> <http://arteika:/solr/logan/query> HTTP/1.1
> content-type: application/json
>
> {
> "query"  : "*:*",
> "limit"  : 0,
> "facet": {
> "t" : {
> "type":  "terms",
> "field": "fcomp",
> "sort":  "index",
>
> "facet": {
> "t_buckets": {
> "type":  "range",
> "field": "t",
> "sort": { "t": "asc" },
> "start": "2018-05-02T17:00:00.000Z",
> "end":   "2020-11-16T21:00:00.000Z",
> "gap":   "+1HOUR"
> }
> }
> },
> }
> }
>
> not surprisingly, it takes a bit to compute the result, so I tried to
> increase the number of threads. How do I do it in JSON format? I tried
> adding
>
> {
> "params": {
> "facet.threads": 8
> },
> "query"  : "*:*",
> ...
> }
>
> and checked the jstack  of the solr java process, but I still see only
> one thread working.  Can I configure params through the params section?
>
> I also tried
>
> {
> "query"  : "*:*",
> "limit"  : 0,
> "facet": {
> "t" : {
> "type":  "terms",
> "field": "fcomp",
> "sort":  "index",
>
> "facet": {
> "t_buckets": {
> "type":  "range",
> "field": "t",
> "sort": { "t": "asc" },
> "start": "2018-05-02T17:00:00.000Z",
> "end":   "2020-11-16T21:00:00.000Z",
> "gap":   "+1HOUR"
> }
> },
> "threads":8
> },
> }
> }
>
> but this ran in one thread as well. Can I influence the number of threads
> in the "facet" section of JSON?
>
> Cheers,
> Arturas
>

increasing number of threads for faceting in JSON format

[Arturas Mazeika]

Hi Solr-Users,

I am trying to better understand the solr capabilities, how one can
formulate queries in JSON format as well as tweak parameters. Currently I
have a logs collection (ca 6GB large) with a dozen of attributes running in
single server mode (F:\solr_deployment\solr-8.7.0\bin\solr.cmd start -h
localhost -p  -m 4g)

I am playing with faceting functionality in solr and query a couple of
attributes there. My typical query is:

GET http://localhost:/solr/db/query
<http://arteika:/solr/logan/query> HTTP/1.1
content-type: application/json

{
"query"  : "*:*",
"limit"  : 0,
"facet": {
"t" : {
"type":  "terms",
"field": "fcomp",
"sort":  "index",

"facet": {
"t_buckets": {
"type":  "range",
"field": "t",
"sort": { "t": "asc" },
"start": "2018-05-02T17:00:00.000Z",
"end":   "2020-11-16T21:00:00.000Z",
"gap":   "+1HOUR"
}
}
},
}
}

not surprisingly, it takes a bit to compute the result, so I tried to
increase the number of threads. How do I do it in JSON format? I tried
adding

{
"params": {
"facet.threads": 8
},
"query"  : "*:*",
...
}

and checked the jstack  of the solr java process, but I still see only
one thread working.  Can I configure params through the params section?

I also tried

{
"query"  : "*:*",
"limit"  : 0,
"facet": {
"t" : {
"type":  "terms",
"field": "fcomp",
"sort":  "index",

"facet": {
"t_buckets": {
"type":  "range",
"field": "t",
"sort": { "t": "asc" },
"start": "2018-05-02T17:00:00.000Z",
"end":   "2020-11-16T21:00:00.000Z",
"gap":   "+1HOUR"
}
},
"threads":8
},
}
}

but this ran in one thread as well. Can I influence the number of threads
in the "facet" section of JSON?

Cheers,
Arturas

Re: Faceting: !terms vs mincount precedence

[Jason Gerlowski]

Thanks for the context David - I didn't realize this was built as an
internal mechanism and then documented later on.  A few other thoughts
below:

> {!terms}, it suggests a reference to the TermsQParser, but when you write 
> {!terms=a,b,c} it suggests local-params
I agree that the two are easy to confuse.  Apologies for abbreviating
it at points in my earlier email - I was doing it for brevity and
didn't intend the confusion.

> I think that "terms" local-param to faceting was a purely internal thing that 
> wasn't documented
That may be.  But I disagree that it shouldn't've been documented in
the first place.  Digging into this has cost me a good bit of time,
and even now maybe I've got more digging to do, maybe a bug to fix,
etc.  But without someone's (Christine's?) documentation I'd be even
worse off, without any idea that this "terms" local-params support
exists at all.  The documentation even mentions that "terms" doesn't
work well with some other faceting params.  The details could be a bit
fuller, but the warning *is* there.  So I don't find any fault with
documenting this sort of stuff - especially when it gives warnings
about potential limitations.

Anyway, still hoping someone else might chime in with a slick
workaround or something.  But it does look at this point like I'll
have to go another route or put in some effort myself.

Jason

On Tue, Nov 17, 2020 at 3:41 PM David Smiley  wrote:
>
> This is confusing because when you write {!terms}, it suggests a reference
> to the TermsQParser, but when you write {!terms=a,b,c} it suggests
> local-params, with key "terms" and value "a,b,c" -- entirely different
> things.  I think that "terms" local-param to faceting was a purely internal
> thing that wasn't documented; it existed as an internal implementation
> detail.  Then someone (I think Christine, if not then Mikhail) observed it
> wasn't documented, and added some basic docs.  Now you come along and try
> to use it with other things that unsurprisingly it just wasn't designed
> for.  That's my estimation of the matter... and *if* true, illustrates that
> maybe some internal params should stay internal and don't need to be
> publicly documented.  I confess I've used that faceting local-param in an
> app once before too; it's useful.  I know my response isn't a direct answer
> to your question RE mincount... perhaps it can be made to work?
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Tue, Nov 17, 2020 at 8:21 AM Jason Gerlowski 
> wrote:
>
> > Hey all,
> >
> > I was using the {!terms} local parameter on some traditional field
> > facets to make sure particular values were returned.
> >
> > e.g.
> > facet=true={!terms='fantasy,scifi,mystery'}genre_s_s.facet.mincount=2
> >
> > On single-shard collections in 8.6.3 this worked as I expected -
> > "fantasy", "scifi", and "mystery" were the only 3 field values
> > returned, and "mystery" was returned despite its count value being
> > less than the specified "mincount".  But on a multi-shard collection
> > "mystery" isn't returned (presumably because a "mincount" check
> > filters out the values on the facet aggregator node).
> >
> > What are the expected semantics when "{!terms}" and "mincount" are
> > used together?  Should mincount filter out values in {!terms}, or
> > should those values be excluded from any mincount filtering?  The
> > behavior is clearly inconsistent between single and multi-shard, so it
> > deserves a JIRA either way.  Just trying to figure out what the
> > expected behavior is.
> >
> > Best,
> >
> > Jason
> >

Re: Faceting: !terms vs mincount precedence

[David Smiley]

This is confusing because when you write {!terms}, it suggests a reference
to the TermsQParser, but when you write {!terms=a,b,c} it suggests
local-params, with key "terms" and value "a,b,c" -- entirely different
things.  I think that "terms" local-param to faceting was a purely internal
thing that wasn't documented; it existed as an internal implementation
detail.  Then someone (I think Christine, if not then Mikhail) observed it
wasn't documented, and added some basic docs.  Now you come along and try
to use it with other things that unsurprisingly it just wasn't designed
for.  That's my estimation of the matter... and *if* true, illustrates that
maybe some internal params should stay internal and don't need to be
publicly documented.  I confess I've used that faceting local-param in an
app once before too; it's useful.  I know my response isn't a direct answer
to your question RE mincount... perhaps it can be made to work?

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Tue, Nov 17, 2020 at 8:21 AM Jason Gerlowski 
wrote:

> Hey all,
>
> I was using the {!terms} local parameter on some traditional field
> facets to make sure particular values were returned.
>
> e.g.
> facet=true={!terms='fantasy,scifi,mystery'}genre_s_s.facet.mincount=2
>
> On single-shard collections in 8.6.3 this worked as I expected -
> "fantasy", "scifi", and "mystery" were the only 3 field values
> returned, and "mystery" was returned despite its count value being
> less than the specified "mincount".  But on a multi-shard collection
> "mystery" isn't returned (presumably because a "mincount" check
> filters out the values on the facet aggregator node).
>
> What are the expected semantics when "{!terms}" and "mincount" are
> used together?  Should mincount filter out values in {!terms}, or
> should those values be excluded from any mincount filtering?  The
> behavior is clearly inconsistent between single and multi-shard, so it
> deserves a JIRA either way.  Just trying to figure out what the
> expected behavior is.
>
> Best,
>
> Jason
>

Faceting: !terms vs mincount precedence

[Jason Gerlowski]

Hey all,

I was using the {!terms} local parameter on some traditional field
facets to make sure particular values were returned.

e.g. 
facet=true={!terms='fantasy,scifi,mystery'}genre_s_s.facet.mincount=2

On single-shard collections in 8.6.3 this worked as I expected -
"fantasy", "scifi", and "mystery" were the only 3 field values
returned, and "mystery" was returned despite its count value being
less than the specified "mincount".  But on a multi-shard collection
"mystery" isn't returned (presumably because a "mincount" check
filters out the values on the facet aggregator node).

What are the expected semantics when "{!terms}" and "mincount" are
used together?  Should mincount filter out values in {!terms}, or
should those values be excluded from any mincount filtering?  The
behavior is clearly inconsistent between single and multi-shard, so it
deserves a JIRA either way.  Just trying to figure out what the
expected behavior is.

Best,

Jason

Re: Faceting on indexed=false stored=false docValues=true fields

[uyilmaz]

Sorry, correction, taking "the" time

On Mon, 19 Oct 2020 22:18:30 +0300
uyilmaz  wrote:

> Thanks for taking time to write a detailed answer.
> 
> We use Solr to both store our data and to perform aggregations, using 
> faceting or streaming expressions. When required analysis is too complex to 
> do in Solr, we export large query results from Solr to a more capable 
> analysis tool.
> 
> So I guess all fields need to be docValues="true", because export handler and 
> streaming both require fields to have docValues, and even if I won't use a 
> field in queries or facets, it should be in available to read in result set. 
> Fields that won't be searched or faceted can be (indexed=false stored=false 
> docValues=true) right?
> 
> --uyilmaz
> 
> 
> On Mon, 19 Oct 2020 14:14:27 -0400
> Michael Gibney  wrote:
> 
> > As you've observed, it is indeed possible to facet on fields with
> > docValues=true, indexed=false; but in almost all cases you should
> > probably set indexed=true. 1. for distributed facet count refinement,
> > the "indexed" approach is used to look up counts by value; 2. assuming
> > you're wanting to do something usual, e.g. allow users to apply
> > filters based on facet counts, the filter application would use the
> > "indexed" approach as well. Where indexed=false, if either filtering
> > or distributed refinement is attempted, I'm not 100% sure what
> > happens. It might fail, or lead to inconsistent results, or attempt to
> > look up results via the equivalent of a "table scan" over docValues (I
> > think the last of these is what actually happens, fwiw) ... but none
> > of these options is likely desirable.
> > 
> > Michael
> > 
> > On Mon, Oct 19, 2020 at 1:42 PM uyilmaz  wrote:
> > >
> > > Thanks! This also contributed to my confusion:
> > >
> > > https://lucene.apache.org/solr/guide/8_4/faceting.html#field-value-faceting-parameters
> > >
> > > "If you want Solr to perform both analysis (for searching) and faceting 
> > > on the full literal strings, use the copyField directive in your Schema 
> > > to create two versions of the field: one Text and one String. Make sure 
> > > both are indexed="true"."
> > >
> > > On Mon, 19 Oct 2020 13:08:00 -0400
> > > Alexandre Rafalovitch  wrote:
> > >
> > > > I think this is all explained quite well in the Ref Guide:
> > > > https://lucene.apache.org/solr/guide/8_6/docvalues.html
> > > >
> > > > DocValues is a different way to index/store values. Faceting is a
> > > > primary use case where docValues are better than what 'indexed=true'
> > > > gives you.
> > > >
> > > > Regards,
> > > >Alex.
> > > >
> > > > On Mon, 19 Oct 2020 at 12:51, uyilmaz  
> > > > wrote:
> > > > >
> > > > >
> > > > > Hey all,
> > > > >
> > > > > From my little experiments, I see that (if I didn't make a stupid 
> > > > > mistake) we can facet on fields marked as both indexed and stored 
> > > > > being false:
> > > > >
> > > > >  > > > > indexed="false" stored="false" docValues="true"/>
> > > > >
> > > > > I'm suprised by this, I thought I would need to index it. Can you 
> > > > > confirm this?
> > > > >
> > > > > Regards
> > > > >
> > > > > --
> > > > > uyilmaz 
> > >
> > >
> > > --
> > > uyilmaz 
> 
> 
> -- 
> uyilmaz 


-- 
uyilmaz 

Re: Faceting on indexed=false stored=false docValues=true fields

[uyilmaz]

Thanks for taking time to write a detailed answer.

We use Solr to both store our data and to perform aggregations, using faceting 
or streaming expressions. When required analysis is too complex to do in Solr, 
we export large query results from Solr to a more capable analysis tool.

So I guess all fields need to be docValues="true", because export handler and 
streaming both require fields to have docValues, and even if I won't use a 
field in queries or facets, it should be in available to read in result set. 
Fields that won't be searched or faceted can be (indexed=false stored=false 
docValues=true) right?

--uyilmaz


On Mon, 19 Oct 2020 14:14:27 -0400
Michael Gibney  wrote:

> As you've observed, it is indeed possible to facet on fields with
> docValues=true, indexed=false; but in almost all cases you should
> probably set indexed=true. 1. for distributed facet count refinement,
> the "indexed" approach is used to look up counts by value; 2. assuming
> you're wanting to do something usual, e.g. allow users to apply
> filters based on facet counts, the filter application would use the
> "indexed" approach as well. Where indexed=false, if either filtering
> or distributed refinement is attempted, I'm not 100% sure what
> happens. It might fail, or lead to inconsistent results, or attempt to
> look up results via the equivalent of a "table scan" over docValues (I
> think the last of these is what actually happens, fwiw) ... but none
> of these options is likely desirable.
> 
> Michael
> 
> On Mon, Oct 19, 2020 at 1:42 PM uyilmaz  wrote:
> >
> > Thanks! This also contributed to my confusion:
> >
> > https://lucene.apache.org/solr/guide/8_4/faceting.html#field-value-faceting-parameters
> >
> > "If you want Solr to perform both analysis (for searching) and faceting on 
> > the full literal strings, use the copyField directive in your Schema to 
> > create two versions of the field: one Text and one String. Make sure both 
> > are indexed="true"."
> >
> > On Mon, 19 Oct 2020 13:08:00 -0400
> > Alexandre Rafalovitch  wrote:
> >
> > > I think this is all explained quite well in the Ref Guide:
> > > https://lucene.apache.org/solr/guide/8_6/docvalues.html
> > >
> > > DocValues is a different way to index/store values. Faceting is a
> > > primary use case where docValues are better than what 'indexed=true'
> > > gives you.
> > >
> > > Regards,
> > >Alex.
> > >
> > > On Mon, 19 Oct 2020 at 12:51, uyilmaz  wrote:
> > > >
> > > >
> > > > Hey all,
> > > >
> > > > From my little experiments, I see that (if I didn't make a stupid 
> > > > mistake) we can facet on fields marked as both indexed and stored being 
> > > > false:
> > > >
> > > >  > > > indexed="false" stored="false" docValues="true"/>
> > > >
> > > > I'm suprised by this, I thought I would need to index it. Can you 
> > > > confirm this?
> > > >
> > > > Regards
> > > >
> > > > --
> > > > uyilmaz 
> >
> >
> > --
> > uyilmaz 


-- 
uyilmaz 

Re: Faceting on indexed=false stored=false docValues=true fields

[Walter Underwood]

Hmm. Fields used for faceting will also be used for filtering, which is a kind
of search. Are docValues OK for filtering? I expect they might be slow the
first time, then cached.

wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/  (my blog)

> On Oct 19, 2020, at 11:15 AM, Erick Erickson  wrote:
> 
> uyilmaz:
> 
> Hmm, that _is_ confusing. And inaccurate.
> 
> In this context, it should read something like
> 
> The Text field should have indexed="true" docValues=“false" if used for 
> searching 
> but not faceting and the String field should have indexed="false" 
> docValues=“true"
> if used for faceting but not searching.
> 
> I’ll fix this, thanks for pointing this out.
> 
> Erick
> 
>> On Oct 19, 2020, at 1:42 PM, uyilmaz  wrote:
>> 
>> Thanks! This also contributed to my confusion:
>> 
>> https://lucene.apache.org/solr/guide/8_4/faceting.html#field-value-faceting-parameters
>> 
>> "If you want Solr to perform both analysis (for searching) and faceting on 
>> the full literal strings, use the copyField directive in your Schema to 
>> create two versions of the field: one Text and one String. Make sure both 
>> are indexed="true"."
>> 
>> On Mon, 19 Oct 2020 13:08:00 -0400
>> Alexandre Rafalovitch  wrote:
>> 
>>> I think this is all explained quite well in the Ref Guide:
>>> https://lucene.apache.org/solr/guide/8_6/docvalues.html
>>> 
>>> DocValues is a different way to index/store values. Faceting is a
>>> primary use case where docValues are better than what 'indexed=true'
>>> gives you.
>>> 
>>> Regards,
>>>  Alex.
>>> 
>>> On Mon, 19 Oct 2020 at 12:51, uyilmaz  wrote:
>>>> 
>>>> 
>>>> Hey all,
>>>> 
>>>> From my little experiments, I see that (if I didn't make a stupid mistake) 
>>>> we can facet on fields marked as both indexed and stored being false:
>>>> 
>>>> >>> stored="false" docValues="true"/>
>>>> 
>>>> I'm suprised by this, I thought I would need to index it. Can you confirm 
>>>> this?
>>>> 
>>>> Regards
>>>> 
>>>> --
>>>> uyilmaz 
>> 
>> 
>> -- 
>> uyilmaz 
> 

Re: Faceting on indexed=false stored=false docValues=true fields

[Erick Erickson]

uyilmaz:

Hmm, that _is_ confusing. And inaccurate.

In this context, it should read something like

The Text field should have indexed="true" docValues=“false" if used for 
searching 
but not faceting and the String field should have indexed="false" 
docValues=“true"
if used for faceting but not searching.

I’ll fix this, thanks for pointing this out.

Erick

> On Oct 19, 2020, at 1:42 PM, uyilmaz  wrote:
> 
> Thanks! This also contributed to my confusion:
> 
> https://lucene.apache.org/solr/guide/8_4/faceting.html#field-value-faceting-parameters
> 
> "If you want Solr to perform both analysis (for searching) and faceting on 
> the full literal strings, use the copyField directive in your Schema to 
> create two versions of the field: one Text and one String. Make sure both are 
> indexed="true"."
> 
> On Mon, 19 Oct 2020 13:08:00 -0400
> Alexandre Rafalovitch  wrote:
> 
>> I think this is all explained quite well in the Ref Guide:
>> https://lucene.apache.org/solr/guide/8_6/docvalues.html
>> 
>> DocValues is a different way to index/store values. Faceting is a
>> primary use case where docValues are better than what 'indexed=true'
>> gives you.
>> 
>> Regards,
>>   Alex.
>> 
>> On Mon, 19 Oct 2020 at 12:51, uyilmaz  wrote:
>>> 
>>> 
>>> Hey all,
>>> 
>>> From my little experiments, I see that (if I didn't make a stupid mistake) 
>>> we can facet on fields marked as both indexed and stored being false:
>>> 
>>> >> stored="false" docValues="true"/>
>>> 
>>> I'm suprised by this, I thought I would need to index it. Can you confirm 
>>> this?
>>> 
>>> Regards
>>> 
>>> --
>>> uyilmaz 
> 
> 
> -- 
> uyilmaz 

Re: Faceting on indexed=false stored=false docValues=true fields

[Michael Gibney]

As you've observed, it is indeed possible to facet on fields with
docValues=true, indexed=false; but in almost all cases you should
probably set indexed=true. 1. for distributed facet count refinement,
the "indexed" approach is used to look up counts by value; 2. assuming
you're wanting to do something usual, e.g. allow users to apply
filters based on facet counts, the filter application would use the
"indexed" approach as well. Where indexed=false, if either filtering
or distributed refinement is attempted, I'm not 100% sure what
happens. It might fail, or lead to inconsistent results, or attempt to
look up results via the equivalent of a "table scan" over docValues (I
think the last of these is what actually happens, fwiw) ... but none
of these options is likely desirable.

Michael

On Mon, Oct 19, 2020 at 1:42 PM uyilmaz  wrote:
>
> Thanks! This also contributed to my confusion:
>
> https://lucene.apache.org/solr/guide/8_4/faceting.html#field-value-faceting-parameters
>
> "If you want Solr to perform both analysis (for searching) and faceting on 
> the full literal strings, use the copyField directive in your Schema to 
> create two versions of the field: one Text and one String. Make sure both are 
> indexed="true"."
>
> On Mon, 19 Oct 2020 13:08:00 -0400
> Alexandre Rafalovitch  wrote:
>
> > I think this is all explained quite well in the Ref Guide:
> > https://lucene.apache.org/solr/guide/8_6/docvalues.html
> >
> > DocValues is a different way to index/store values. Faceting is a
> > primary use case where docValues are better than what 'indexed=true'
> > gives you.
> >
> > Regards,
> >Alex.
> >
> > On Mon, 19 Oct 2020 at 12:51, uyilmaz  wrote:
> > >
> > >
> > > Hey all,
> > >
> > > From my little experiments, I see that (if I didn't make a stupid 
> > > mistake) we can facet on fields marked as both indexed and stored being 
> > > false:
> > >
> > >  > > stored="false" docValues="true"/>
> > >
> > > I'm suprised by this, I thought I would need to index it. Can you confirm 
> > > this?
> > >
> > > Regards
> > >
> > > --
> > > uyilmaz 
>
>
> --
> uyilmaz 

Re: Faceting on indexed=false stored=false docValues=true fields

[uyilmaz]

Thanks! This also contributed to my confusion:

https://lucene.apache.org/solr/guide/8_4/faceting.html#field-value-faceting-parameters

"If you want Solr to perform both analysis (for searching) and faceting on the 
full literal strings, use the copyField directive in your Schema to create two 
versions of the field: one Text and one String. Make sure both are 
indexed="true"."

On Mon, 19 Oct 2020 13:08:00 -0400
Alexandre Rafalovitch  wrote:

> I think this is all explained quite well in the Ref Guide:
> https://lucene.apache.org/solr/guide/8_6/docvalues.html
> 
> DocValues is a different way to index/store values. Faceting is a
> primary use case where docValues are better than what 'indexed=true'
> gives you.
> 
> Regards,
>Alex.
> 
> On Mon, 19 Oct 2020 at 12:51, uyilmaz  wrote:
> >
> >
> > Hey all,
> >
> > From my little experiments, I see that (if I didn't make a stupid mistake) 
> > we can facet on fields marked as both indexed and stored being false:
> >
> >  > stored="false" docValues="true"/>
> >
> > I'm suprised by this, I thought I would need to index it. Can you confirm 
> > this?
> >
> > Regards
> >
> > --
> > uyilmaz 


-- 
uyilmaz 

Re: Faceting on indexed=false stored=false docValues=true fields

[Alexandre Rafalovitch]

I think this is all explained quite well in the Ref Guide:
https://lucene.apache.org/solr/guide/8_6/docvalues.html

DocValues is a different way to index/store values. Faceting is a
primary use case where docValues are better than what 'indexed=true'
gives you.

Regards,
   Alex.

On Mon, 19 Oct 2020 at 12:51, uyilmaz  wrote:
>
>
> Hey all,
>
> From my little experiments, I see that (if I didn't make a stupid mistake) we 
> can facet on fields marked as both indexed and stored being false:
>
>  stored="false" docValues="true"/>
>
> I'm suprised by this, I thought I would need to index it. Can you confirm 
> this?
>
> Regards
>
> --
> uyilmaz 

Faceting on indexed=false stored=false docValues=true fields

[uyilmaz]

Hey all,

>From my little experiments, I see that (if I didn't make a stupid mistake) we 
>can facet on fields marked as both indexed and stored being false:



I'm suprised by this, I thought I would need to index it. Can you confirm this?

Regards

-- 
uyilmaz 

Re: Terms faceting and EnumField

[Poornima Ponnuswamy]

I see a defect created for that. But I see its resolved. Am I doing anything 
wrong?

https://issues.apache.org/jira/browse/SOLR-7496


QUERY
http://localhost:8983/solr/activity01us/select?
={ServiceRequestTypeCode:{type:terms, field:ServiceRequestTypeCode, 
limit:10}}
=on=on=json=*=0

Below is the field and field type that is defined in solr schema.



Below is the configuration for the enum
   

  servicerequestcorrective
  servicerequestplanned
  servicerequestinstallationandupgrade
  servicerequestrecall
  servicerequestother
  servicerequestinquiry
  servicerequestproactive
  servicerequestsystemupdate
  servicerequesticenteradmin
  servicerequestonwatch
  servicerequestfmi
  servicerequestapplication
   

 I am getting error
"Expected numeric field type 
:ServiceRequestTypeCode{type=ServiceRequestTypeCode,properties=indexed,stored,omitNorms,omitTermFreqAndPositions}"




From: Poornima Ponnuswamy 
Sent: Friday, May 22, 2020 7:01 AM
To: solr-user@lucene.apache.org 
Subject: Terms faceting and EnumField

Hello,

We have solr 6.6 version.
Below is the field and field type that is defined in solr schema.



Below is the configuration for the enum
   

  servicerequestcorrective
  servicerequestplanned
  servicerequestinstallationandupgrade
  servicerequestrecall
  servicerequestother
  servicerequestinquiry
  servicerequestproactive
  servicerequestsystemupdate
  servicerequesticenteradmin
  servicerequestonwatch
  servicerequestfmi
  servicerequestapplication
   

When I try to invoke using the below request,

http://localhost:8983/solr/activity01us/select?={ServiceRequestTypeCode:{type:terms<http://localhost:8983/solr/activity01us/select?=%7bServiceRequestTypeCode:%7btype:terms><http://localhost:8983/solr/activity01us/select?={ServiceRequestTypeCode:{type:terms%3Chttp://localhost:8983/solr/activity01us/select?=%7bServiceRequestTypeCode:%7btype:terms%3E>,
 field:ServiceRequestTypeCode, limit:10}}=on=on=json=*

 I am getting error
"Expected numeric field type 
:ServiceRequestTypeCode{type=ServiceRequestTypeCode,properties=indexed,stored,omitNorms,omitTermFreqAndPositions}"

But when I try to do as below it works fine.


http://localhost:8983/solr/activity01us/select?facet.field=ServiceRequestTypeCode=on=on=*:*=json

I would like to use json facet as it would help me in subfaceting.

Any help would be appreciated

Terms faceting and EnumField

[Poornima Ponnuswamy]

Hello,

We have solr 6.6 version.
Below is the field and field type that is defined in solr schema.



Below is the configuration for the enum
   

  servicerequestcorrective
  servicerequestplanned
  servicerequestinstallationandupgrade
  servicerequestrecall
  servicerequestother
  servicerequestinquiry
  servicerequestproactive
  servicerequestsystemupdate
  servicerequesticenteradmin
  servicerequestonwatch
  servicerequestfmi
  servicerequestapplication
   

When I try to invoke using the below request,

http://localhost:8983/solr/activity01us/select?={ServiceRequestTypeCode:{type:terms,
 field:ServiceRequestTypeCode, limit:10}}=on=on=json=*

 I am getting error
"Expected numeric field type 
:ServiceRequestTypeCode{type=ServiceRequestTypeCode,properties=indexed,stored,omitNorms,omitTermFreqAndPositions}"

But when I try to do as below it works fine.


http://localhost:8983/solr/activity01us/select?facet.field=ServiceRequestTypeCode=on=on=*:*=json

I would like to use json facet as it would help me in subfaceting.

Any help would be appreciated

Re: json faceting - Terms faceting and EnumField

[Ponnuswamy, Poornima (GE Healthcare)]

Can anyone provide some light on the issue I am having?. Thanks!

On 5/20/20, 4:55 PM, "Ponnuswamy, Poornima (GE Healthcare)" 
 wrote:

Hello,

We have solr 6.6 version.
Below is the field and field type that is defined in solr schema.



Below is the configuration for the enum
   

  servicerequestcorrective
  servicerequestplanned
  servicerequestinstallationandupgrade
  servicerequestrecall
  servicerequestother
  servicerequestinquiry
  servicerequestproactive
  servicerequestsystemupdate
  servicerequesticenteradmin
  servicerequestonwatch
  servicerequestfmi
  servicerequestapplication
   

When I try to invoke using the below call, I am getting error

http://localhost:8983/solr/activity01us/select?={ServiceRequestTypeCode:{type:terms,
 field:ServiceRequestTypeCode, limit:10}}=on=on=json=*
"Expected numeric field type 
:ServiceRequestTypeCode{type=ServiceRequestTypeCode,properties=indexed,stored,omitNorms,omitTermFreqAndPositions}"

But when I try to do as below it works fine.


http://localhost:8983/solr/activity01us/select?facet.field=ServiceRequestTypeCode=on=on=*:*=json

I would like to use json facet as it would help me in subfaceting.

Any help would be appreciated


Thanks,
Poornima

json faceting - Terms faceting and EnumField

[Ponnuswamy, Poornima (GE Healthcare)]

Hello,

We have solr 6.6 version.
Below is the field and field type that is defined in solr schema.



Below is the configuration for the enum
   

  servicerequestcorrective
  servicerequestplanned
  servicerequestinstallationandupgrade
  servicerequestrecall
  servicerequestother
  servicerequestinquiry
  servicerequestproactive
  servicerequestsystemupdate
  servicerequesticenteradmin
  servicerequestonwatch
  servicerequestfmi
  servicerequestapplication
   

When I try to invoke using the below call, I am getting error
http://localhost:8983/solr/activity01us/select?={ServiceRequestTypeCode:{type:terms,
 field:ServiceRequestTypeCode, limit:10}}=on=on=json=*
"Expected numeric field type 
:ServiceRequestTypeCode{type=ServiceRequestTypeCode,properties=indexed,stored,omitNorms,omitTermFreqAndPositions}"

But when I try to do as below it works fine.

http://localhost:8983/solr/activity01us/select?facet.field=ServiceRequestTypeCode=on=on=*:*=json

I would like to use json facet as it would help me in subfaceting.

Any help would be appreciated


Thanks,
Poornima

Re: multivalue faceting term optimization

[Jörn Franke]

hll stands for https://en.wikipedia.org/wiki/HyperLogLog

You will not get the exact distinct count, but a distinct count very close to 
the real number. It is very fast and memory efficient for large number of 
distinct values.

> Am 10.03.2020 um 00:25 schrieb Nicolas Paris :
> 
> 
> Erick Erickson  writes:
>> Have you looked at the HyperLogLog stuff? Here’s at least a mention of
>> it: https://lucene.apache.org/solr/guide/8_4/the-stats-component.html
> 
> I am used to hll in the context of count distinct values -- cardinality.
> 
> I have to admit that section 
> https://lucene.apache.org/solr/guide/8_4/the-stats-component.html#local-parameters-with-the-stats-component
> is about hll and facets, but I am not sure that really meet the use
> case. I also have to admit that part is quite cryptic to me.
> 
> 
> -- 
> nicolas paris

Re: multivalue faceting term optimization

[Nicolas Paris]

Erick Erickson  writes:
> Have you looked at the HyperLogLog stuff? Here’s at least a mention of
> it: https://lucene.apache.org/solr/guide/8_4/the-stats-component.html

I am used to hll in the context of count distinct values -- cardinality.

I have to admit that section 
https://lucene.apache.org/solr/guide/8_4/the-stats-component.html#local-parameters-with-the-stats-component
is about hll and facets, but I am not sure that really meet the use
case. I also have to admit that part is quite cryptic to me.


-- 
nicolas paris

Re: multivalue faceting term optimization

[Nicolas Paris]

Toke Eskildsen  writes:
> JSON faceting allows you to skip the fine counting with the parameter
> refine: 

I also tried the facet.refine parameter, but didn't notice any improvement.


>> I am wondering how I could filter the documents to get approximate
>> facets ?
>
> Clunky idea: Introduce a hash field for each document. [...]
> [...]you could also create fields with random values

That's a pragmatic solution. Two steps:
1. get the count, hightlight and first matches
2. depending on the count, filter based on random/hash values

BTW I wonder if the first step will be cached, as to get highlights I
cannot use FQ, but Q. And the latter is not meant to cache the
results. So this might lead to duplicate the effort isn'it ?


> It might help to have everything in a single shard, to avoid the
> secondary fine count. But your index is rather large

Yes, it's large, and growing from 1M each month. Merging in one shard is
not an option.

However, I suppose I should be able to ask the facet to one shard only
if the count is above a threshold ? This would reduce the number of
document by ~4 and avoid secondary fine count. That maybe better than
subsetting with extra random fields

-- 
nicolas paris

Re: multivalue faceting term optimization

[Erick Erickson]

Have you looked at the HyperLogLog stuff? Here’s at least a mention of it: 
https://lucene.apache.org/solr/guide/8_4/the-stats-component.html



Best,
Erick

> On Mar 9, 2020, at 02:39, Nicolas Paris  wrote:
> 
> Hello,
> 
> 
> Environment:
> - SolrCloud 8.4.1
> - 4 shards with xmx = 120GO and ssd disks
> - 50M documents / 40GO physical per shard
> - mainly large texts fields and also, one multivalue/docvalue/indexed string
> list of 15 values per document
> 
> 
> Goal:
> I want to provide terms facet on a string multivalue field. This offers
> the client to build dynamic word cloud depending on filter queries. The
> words within the array are extracted with TFIDF from large raw texts
> from neightbourg fields.
> 
> 
> Behavior:
> The computing time is below 2 seconds when the FQ query is selective
> enough (<2M). However it results as a timeout when the FQ finds > 2M documents
> 
> 
> Question:
> How to improve brute performances ?
> I tried:
> - facet.limit / facet.threads / facet.method
> - limiting the multivalue size (from 50 elements per documents to 15)
> Is ther any parameter I am missing ?
> 
> 
> Thought:
> If there is now way to faster performances for the brute task, I guess I
> could artificially limit the FQ under 2M for all queries by getting a
> sample (I don't really care having more than 2M documents to build the
> word cloud).
> I am wondering how I could filter the documents to get approximate facets ?
> 
> 
> Thanks !
> 
> 
> -- 
> nicolas paris

Re: multivalue faceting term optimization

[Toke Eskildsen]

On Mon, 2020-03-09 at 10:39 +0100, Nicolas Paris wrote:
> I want to provide terms facet on a string multivalue field.
> ...
> How to improve brute performances ?

It might help to have everything in a single shard, to avoid the
secondary fine count. But your index is rather large for single-shard
so that might have a negative impact on overall speed.

JSON faceting allows you to skip the fine counting with the parameter
refine: 
https://lucene.apache.org/solr/guide/8_4/json-facet-api.html#terms-facet
Should be easy to try.

> I am wondering how I could filter the documents to get approximate
> facets ?

Clunky idea: Introduce a hash field for each document. When you need
the heavy facet call, start by a search just to count the number of
documents. If is is too high, add a prefix-filter for the hash-field
with a random hex value.

1M hits:
q=foo
-> Facets for 1M documents

10M hits:
q=foo=hash:1*
-> Facets for 620K documents (10M/16)

100M hits:
q=foo=hash:ab*
-> Facets for 390K documents (100M/256)

If you want it more fine-grained to hit closer to your ~2M limit, you
could add a bit more filter logic:

100M hits:
q=foo=hash:00* OR hash:01* OR hash:02* OR hash:03* OR hash:04*
-> Facets for 1950K documents (100M/256 * 5)

Prefix queries might prove to be too expensive, so you could also
create fields with random values from 0-9, 0-99, 0-999 etc. and do
exact match filtering on those to get the number of hits down.


- Toke Eskildsen, Royal Danish Library

multivalue faceting term optimization

[Nicolas Paris]

Hello,


Environment:
- SolrCloud 8.4.1
- 4 shards with xmx = 120GO and ssd disks
- 50M documents / 40GO physical per shard
- mainly large texts fields and also, one multivalue/docvalue/indexed string
list of 15 values per document


Goal:
I want to provide terms facet on a string multivalue field. This offers
the client to build dynamic word cloud depending on filter queries. The
words within the array are extracted with TFIDF from large raw texts
from neightbourg fields.


Behavior:
The computing time is below 2 seconds when the FQ query is selective
enough (<2M). However it results as a timeout when the FQ finds > 2M documents


Question:
How to improve brute performances ?
I tried:
- facet.limit / facet.threads / facet.method
- limiting the multivalue size (from 50 elements per documents to 15)
Is ther any parameter I am missing ?


Thought:
If there is now way to faster performances for the brute task, I guess I
could artificially limit the FQ under 2M for all queries by getting a
sample (I don't really care having more than 2M documents to build the
word cloud).
I am wondering how I could filter the documents to get approximate facets ?


Thanks !


-- 
nicolas paris

Re: NoClassDefFoundError - Faceting on 8.2.0

[Erick Erickson]

My first guess is that you have multiple or out-of-date jars in your classpath 
on those machines.

Best,
Erick

> On Feb 5, 2020, at 5:53 PM, Joe Obernberger  
> wrote:
> 
> Hi All - getting this error intermittently on a solr cloud cluster.  
> Sometimes the heatmap generation works, sometimes not.  I tracked it down to 
> some of the nodes are reporting this error:
> 
> null:java.lang.RuntimeException: java.lang.NoClassDefFoundError: Could not 
> initialize class org.apache.solr.search.facet.FacetHeatmap$PngHelper
>   at org.apache.solr.servlet.HttpSolrCall.sendError(HttpSolrCall.java:733)
>   at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:591)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:423)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:350)
>   at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
>   at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
>   at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1347)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1678)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1249)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
>   at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)
>   at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:152)
>   at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
>   at 
> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
>   at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
>   at org.eclipse.jetty.server.Server.handle(Server.java:505)
>   at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370)
>   at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:311)
>   at 
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
>   at 
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
>   at 
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
>   at 
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
>   at 
> org.eclipse.jetty.http2.HTTP2Connection.produce(HTTP2Connection.java:170)
>   at 
> org.eclipse.jetty.http2.HTTP2Connection.onFillable(HTTP2Connection.java:125)
>   at 
> org.eclipse.jetty.http2.HTTP2Connection$FillableCallback.succeeded(HTTP2Connection.java:348)
>   at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
>   at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
>   at 
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
>   at 
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
>   at 
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
>   at 
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
>   at 
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
>   at 
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:781)
>   at 
> org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:917)
>   at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.NoClassDefFoundError: Could not initialize class 
> org.apache.solr.search.facet.FacetHeatmap$PngHelper
>   at 
> org.apache.solr.search.facet.FacetHeatmap.asPngBytes(FacetHeatmap.java:406)
>   at 
> org.apache.solr.search.facet.FacetHeatmap.formatCountsVal(FacetHeatmap.java:295)
>   at 
> 

NoClassDefFoundError - Faceting on 8.2.0

[Joe Obernberger]

Hi All - getting this error intermittently on a solr cloud cluster.  
Sometimes the heatmap generation works, sometimes not.  I tracked it 
down to some of the nodes are reporting this error:


null:java.lang.RuntimeException: java.lang.NoClassDefFoundError: Could not 
initialize class org.apache.solr.search.facet.FacetHeatmap$PngHelper
at org.apache.solr.servlet.HttpSolrCall.sendError(HttpSolrCall.java:733)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:591)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:423)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:350)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)
at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711)
at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1347)
at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1678)
at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1249)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)
at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:152)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at 
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:505)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370)
at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:311)
at 
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
at 
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
at 
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at 
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
at 
org.eclipse.jetty.http2.HTTP2Connection.produce(HTTP2Connection.java:170)
at 
org.eclipse.jetty.http2.HTTP2Connection.onFillable(HTTP2Connection.java:125)
at 
org.eclipse.jetty.http2.HTTP2Connection$FillableCallback.succeeded(HTTP2Connection.java:348)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
at 
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
at 
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
at 
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at 
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:781)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:917)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NoClassDefFoundError: Could not initialize class 
org.apache.solr.search.facet.FacetHeatmap$PngHelper
at 
org.apache.solr.search.facet.FacetHeatmap.asPngBytes(FacetHeatmap.java:406)
at 
org.apache.solr.search.facet.FacetHeatmap.formatCountsVal(FacetHeatmap.java:295)
at 
org.apache.solr.search.facet.FacetHeatmap.access$500(FacetHeatmap.java:68)
at 
org.apache.solr.search.facet.FacetHeatmap$FacetHeatmapProcessor.process(FacetHeatmap.java:251)
at 
org.apache.solr.search.facet.FacetRequest.process(FacetRequest.java:401)
at 

Re: Excluding a block join filter query during faceting

[Srijan]

Oh wow, I feel stupid. I was trying {*!parent* which="my_doc_type:Parent"
*!tag*=test}. Thanks a lot.

On Wed, Dec 4, 2019, 03:11 Mikhail Khludnev  wrote:

> Why do you think it doesn't work?
> ={!parent tag=test
> which="my_doc_type:Parent"}child_doc_some_field:("30")
>
> On Wed, Dec 4, 2019 at 12:38 AM Srijan  wrote:
>
> > I was wondering if anyone has encountered this problem.
> >
> > I have a parent block join query to return parent documents when child
> > documents are matched.
> > Eg:
> > q=
> > ={!parent which="my_doc_type:Parent"}child_doc_some_field:("30")
> >
> > I now want to facet on certain parent field but want to exclude the above
> > filter query condition entirely. If I had a normal filter query,
> > fq={!tag=test}parent_doc_field1:("30") then I could use that as my
> exclude
> > tag while faceting.
> > facet.field={!ex=test}parent_doc_field2&...
> >
> > But turns out I cannot do that with a block join filter query. Is there
> > anyway I can achieve this? JSON faceting with domain filter capability
> will
> > probably solve my problem but I cannot use JSON faceting at this point.
> >
> > Thanks a lot,
> >
> > Srijan Nepal
> >
>
>
> --
> Sincerely yours
> Mikhail Khludnev
>

Re: Excluding a block join filter query during faceting

[Mikhail Khludnev]

Why do you think it doesn't work?
={!parent tag=test
which="my_doc_type:Parent"}child_doc_some_field:("30")

On Wed, Dec 4, 2019 at 12:38 AM Srijan  wrote:

> I was wondering if anyone has encountered this problem.
>
> I have a parent block join query to return parent documents when child
> documents are matched.
> Eg:
> q=
> ={!parent which="my_doc_type:Parent"}child_doc_some_field:("30")
>
> I now want to facet on certain parent field but want to exclude the above
> filter query condition entirely. If I had a normal filter query,
> fq={!tag=test}parent_doc_field1:("30") then I could use that as my exclude
> tag while faceting.
> facet.field={!ex=test}parent_doc_field2&...
>
> But turns out I cannot do that with a block join filter query. Is there
> anyway I can achieve this? JSON faceting with domain filter capability will
> probably solve my problem but I cannot use JSON faceting at this point.
>
> Thanks a lot,
>
> Srijan Nepal
>


-- 
Sincerely yours
Mikhail Khludnev

Excluding a block join filter query during faceting

[Srijan]

I was wondering if anyone has encountered this problem.

I have a parent block join query to return parent documents when child
documents are matched.
Eg:
q=
={!parent which="my_doc_type:Parent"}child_doc_some_field:("30")

I now want to facet on certain parent field but want to exclude the above
filter query condition entirely. If I had a normal filter query,
fq={!tag=test}parent_doc_field1:("30") then I could use that as my exclude
tag while faceting.
facet.field={!ex=test}parent_doc_field2&...

But turns out I cannot do that with a block join filter query. Is there
anyway I can achieve this? JSON faceting with domain filter capability will
probably solve my problem but I cannot use JSON faceting at this point.

Thanks a lot,

Srijan Nepal

Re: more like this query parser with faceting

[Szűcs Roland]

Thanks David.
This page I was looking for.

Roland

David Hastings  ezt írta (időpont: 2019. aug.
12., H, 20:52):

> should be fine,
> https://cwiki.apache.org/confluence/display/solr/MoreLikeThisHandler
>
> for more info
>
> On Mon, Aug 12, 2019 at 2:49 PM Szűcs Roland 
> wrote:
>
> > Hi David,
> > Thanks the fast reply. Am I right that I can combine fq with mlt only if
> I
> > use more like this as a query parser?
> >
> > Is there a way to achieve the same with mlt as a request handler?
> > Roland
> >
> > David Hastings  ezt írta (időpont: 2019.
> > aug.
> > 12., H, 20:44):
> >
> > > The easiest way will be to pass in a filter query (fq)
> > >
> > > On Mon, Aug 12, 2019 at 2:40 PM Szűcs Roland <
> > szucs.rol...@bookandwalk.hu>
> > > wrote:
> > >
> > > > Hi All,
> > > >
> > > > Is there any tutorial or example how to use more like this
> > functionality
> > > > when we have some other constraints set by the user through faceting
> > > > parameters like price range, or product category for example?
> > > >
> > > > Cheers,
> > > > Roland
> > > >
> > >
> >
>

Re: more like this query parser with faceting

[David Hastings]

should be fine,
https://cwiki.apache.org/confluence/display/solr/MoreLikeThisHandler

for more info

On Mon, Aug 12, 2019 at 2:49 PM Szűcs Roland 
wrote:

> Hi David,
> Thanks the fast reply. Am I right that I can combine fq with mlt only if I
> use more like this as a query parser?
>
> Is there a way to achieve the same with mlt as a request handler?
> Roland
>
> David Hastings  ezt írta (időpont: 2019.
> aug.
> 12., H, 20:44):
>
> > The easiest way will be to pass in a filter query (fq)
> >
> > On Mon, Aug 12, 2019 at 2:40 PM Szűcs Roland <
> szucs.rol...@bookandwalk.hu>
> > wrote:
> >
> > > Hi All,
> > >
> > > Is there any tutorial or example how to use more like this
> functionality
> > > when we have some other constraints set by the user through faceting
> > > parameters like price range, or product category for example?
> > >
> > > Cheers,
> > > Roland
> > >
> >
>

Re: more like this query parser with faceting

[Szűcs Roland]

Hi David,
Thanks the fast reply. Am I right that I can combine fq with mlt only if I
use more like this as a query parser?

Is there a way to achieve the same with mlt as a request handler?
Roland

David Hastings  ezt írta (időpont: 2019. aug.
12., H, 20:44):

> The easiest way will be to pass in a filter query (fq)
>
> On Mon, Aug 12, 2019 at 2:40 PM Szűcs Roland 
> wrote:
>
> > Hi All,
> >
> > Is there any tutorial or example how to use more like this functionality
> > when we have some other constraints set by the user through faceting
> > parameters like price range, or product category for example?
> >
> > Cheers,
> > Roland
> >
>

Re: more like this query parser with faceting

[David Hastings]

The easiest way will be to pass in a filter query (fq)

On Mon, Aug 12, 2019 at 2:40 PM Szűcs Roland 
wrote:

> Hi All,
>
> Is there any tutorial or example how to use more like this functionality
> when we have some other constraints set by the user through faceting
> parameters like price range, or product category for example?
>
> Cheers,
> Roland
>

more like this query parser with faceting

[Szűcs Roland]

Hi All,

Is there any tutorial or example how to use more like this functionality
when we have some other constraints set by the user through faceting
parameters like price range, or product category for example?

Cheers,
Roland

Re: Faceting with Stats

[Erick Erickson]

Thanks for bring closure to this. Yeah, “escaping hell” is something that
happens to us all, something that works in a browser doesn’t work
from SolrJ and neither one may work with curl and……

Pretty often, BTW, I look at the Solr log. It takes a little practice to 
reconstruct the query, but it’s not very hard. Then I work 
backwards…

Best,
Erick

> On Jul 4, 2019, at 11:14 PM, Ahmed Adel  wrote:
> 
> Thanks for your reply! Yes, it turned out to be an issue with the way the
> request was being sent, which was cURL that required special handling and
> escaping of spaces and special characters. Using another client cleared
> this issue and the request below worked perfectly now.
> 
> Best,
> A.
> 
> On Thu, Jul 4, 2019 at 4:53 PM Erick Erickson 
> wrote:
> 
>> Might be a formatting error with my mail client, but the very first line
>> is not well formed.
>> 
>> q: * is incorrect
>> 
>> q=*:*
>> 
>> 
>> 
>> I do not see that example on the page either. Looks like you took the bit
>> that starts with stats=true and mis-typed the q clause.
>> 
>> Best,
>> Erick
>>> On Jul 3, 2019, at 5:08 AM, Ahmed Adel  wrote:
>>> 
>>> Hi,
>>> 
>>> As per the documentation recommendation of using pivot with stats
>> component
>>> instead (
>>> 
>> https://lucene.apache.org/solr/guide/8_1/faceting.html#combining-stats-component-with-pivots
>> ),
>>> replacing the stats options that were previously used with the newer
>> pivot
>>> options as follows:
>>> 
>>> q: *
>>> stats=true
>>> stats.field={!tag=piv1 mean=true}average_rating_f
>>> facet=true
>>> facet.pivot={!stats=piv1}author_s
>>> 
>>> returns the following error:
>>> 
>>> Bad Message 400
>>> reason: Illegal character SPACE=' '
>>> 
>>> This is a syntax issue rather than a logical one, however. Any thoughts
>> of
>>> what could be missing would be appreciated.
>>> 
>>> Thanks,
>>> A. Adel
>>> 
>>> On Tue, Jul 2, 2019 at 4:38 PM Ahmed Adel  wrote:
>>> 
 Hi,
 
 How can stats field value be calculated for top facet values? In other
 words, the following request parameters should return the stats.field
 measures for facets sorted by count:
 
 q: *
 wt: json
 stats: true
 stats.facet: authors_s
 stats.field: average_rating_f
 facet.missing: true
 f.authors_s.facet.sort: count
 
 However, the response is not sorted by facet field count. Is there
 something missing?
 
 Best,
 A.
 
>> 
>> --
> Sent from my iPhone

Re: Faceting with Stats

[Ahmed Adel]

Thanks for your reply! Yes, it turned out to be an issue with the way the
request was being sent, which was cURL that required special handling and
escaping of spaces and special characters. Using another client cleared
this issue and the request below worked perfectly now.

Best,
A.

On Thu, Jul 4, 2019 at 4:53 PM Erick Erickson 
wrote:

> Might be a formatting error with my mail client, but the very first line
> is not well formed.
>
> q: * is incorrect
>
> q=*:*
>
>
>
> I do not see that example on the page either. Looks like you took the bit
> that starts with stats=true and mis-typed the q clause.
>
> Best,
> Erick
> > On Jul 3, 2019, at 5:08 AM, Ahmed Adel  wrote:
> >
> > Hi,
> >
> > As per the documentation recommendation of using pivot with stats
> component
> > instead (
> >
> https://lucene.apache.org/solr/guide/8_1/faceting.html#combining-stats-component-with-pivots
> ),
> > replacing the stats options that were previously used with the newer
> pivot
> > options as follows:
> >
> > q: *
> > stats=true
> > stats.field={!tag=piv1 mean=true}average_rating_f
> > facet=true
> > facet.pivot={!stats=piv1}author_s
> >
> > returns the following error:
> >
> > Bad Message 400
> > reason: Illegal character SPACE=' '
> >
> > This is a syntax issue rather than a logical one, however. Any thoughts
> of
> > what could be missing would be appreciated.
> >
> > Thanks,
> > A. Adel
> >
> > On Tue, Jul 2, 2019 at 4:38 PM Ahmed Adel  wrote:
> >
> >> Hi,
> >>
> >> How can stats field value be calculated for top facet values? In other
> >> words, the following request parameters should return the stats.field
> >> measures for facets sorted by count:
> >>
> >> q: *
> >> wt: json
> >> stats: true
> >> stats.facet: authors_s
> >> stats.field: average_rating_f
> >> facet.missing: true
> >> f.authors_s.facet.sort: count
> >>
> >> However, the response is not sorted by facet field count. Is there
> >> something missing?
> >>
> >> Best,
> >> A.
> >>
>
> --
Sent from my iPhone

Re: Faceting with Stats

[Erick Erickson]

Might be a formatting error with my mail client, but the very first line is not 
well formed. 

q: * is incorrect

q=*:*



I do not see that example on the page either. Looks like you took the bit
that starts with stats=true and mis-typed the q clause.

Best,
Erick
> On Jul 3, 2019, at 5:08 AM, Ahmed Adel  wrote:
> 
> Hi,
> 
> As per the documentation recommendation of using pivot with stats component
> instead (
> https://lucene.apache.org/solr/guide/8_1/faceting.html#combining-stats-component-with-pivots),
> replacing the stats options that were previously used with the newer pivot
> options as follows:
> 
> q: *
> stats=true
> stats.field={!tag=piv1 mean=true}average_rating_f
> facet=true
> facet.pivot={!stats=piv1}author_s
> 
> returns the following error:
> 
> Bad Message 400
> reason: Illegal character SPACE=' '
> 
> This is a syntax issue rather than a logical one, however. Any thoughts of
> what could be missing would be appreciated.
> 
> Thanks,
> A. Adel
> 
> On Tue, Jul 2, 2019 at 4:38 PM Ahmed Adel  wrote:
> 
>> Hi,
>> 
>> How can stats field value be calculated for top facet values? In other
>> words, the following request parameters should return the stats.field
>> measures for facets sorted by count:
>> 
>> q: *
>> wt: json
>> stats: true
>> stats.facet: authors_s
>> stats.field: average_rating_f
>> facet.missing: true
>> f.authors_s.facet.sort: count
>> 
>> However, the response is not sorted by facet field count. Is there
>> something missing?
>> 
>> Best,
>> A.
>> 

Re: Faceting with Stats

[Ahmed Adel]

Hi,

As per the documentation recommendation of using pivot with stats component
instead (
https://lucene.apache.org/solr/guide/8_1/faceting.html#combining-stats-component-with-pivots),
replacing the stats options that were previously used with the newer pivot
options as follows:

q: *
stats=true
stats.field={!tag=piv1 mean=true}average_rating_f
facet=true
facet.pivot={!stats=piv1}author_s

returns the following error:

Bad Message 400
reason: Illegal character SPACE=' '

This is a syntax issue rather than a logical one, however. Any thoughts of
what could be missing would be appreciated.

Thanks,
A. Adel

On Tue, Jul 2, 2019 at 4:38 PM Ahmed Adel  wrote:

> Hi,
>
> How can stats field value be calculated for top facet values? In other
> words, the following request parameters should return the stats.field
> measures for facets sorted by count:
>
> q: *
> wt: json
> stats: true
> stats.facet: authors_s
> stats.field: average_rating_f
> facet.missing: true
> f.authors_s.facet.sort: count
>
> However, the response is not sorted by facet field count. Is there
> something missing?
>
> Best,
> A.
>

Re: Faceting with Stats

[Ahmed Adel]

Hi,

As per the documentation recommendation of using pivot with stats component
instead (
https://lucene.apache.org/solr/guide/8_1/faceting.html#combining-stats-component-with-pivots),
replacing the stats options that were previously used with the newer pivot
options as follows:

q: *
stats=true
stats.field={!tag=piv1 mean=true}average_rating_f
facet=true
facet.pivot={!stats=piv1}author_s

returns the following error:

Bad Message 400
reason: Illegal character SPACE=' '

This is a syntax issue rather than a logical one, however. Any thoughts of
what could be missing would be appreciated.

Thanks,
A. Adel

On Tue, Jul 2, 2019 at 4:38 PM Ahmed Adel  wrote:

> Hi,
>
> How can stats field value be calculated for top facet values? In other
> words, the following request parameters should return the stats.field
> measures for facets sorted by count:
>
> q: *
> wt: json
> stats: true
> stats.facet: authors_s
> stats.field: average_rating_f
> facet.missing: true
> f.authors_s.facet.sort: count
>
> However, the response is not sorted by facet field count. Is there
> something missing?
>
> Best,
> A.
>

Faceting with Stats

[Ahmed Adel]

Hi,

How can stats field value be calculated for top facet values? In other
words, the following request parameters should return the stats.field
measures for facets sorted by count:

q: *
wt: json
stats: true
stats.facet: authors_s
stats.field: average_rating_f
facet.missing: true
f.authors_s.facet.sort: count

However, the response is not sorted by facet field count. Is there
something missing?

Best,
A.

Grouping facet count is not working with JSON Faceting

[Ganesan, VinothKumar]

Hi,

I am trying to use JSON faceting in SOLR 7.3 along with grouping documents and 
noticed that group.facet=true is not working with json.facet but it works with 
flat faceting.

Query Format I used: 
http://localhost:8983/solr/Collection/select?q=*:*=true=2={sc_uf_facetfield:{type:terms,limit:-1,sort:{index:asc},field:facetfield}}=false=false=grouped=1=groupid=true<http://localhost:8983/solr/Collection/select?q=*:*=true=2=%7bsc_uf_facetfield:%7btype:terms,limit:-1,sort:%7bindex:asc%7d,field:facetfield%7d%7d=false=false=grouped=1=groupid=true>

I have 2 documents in my index and both have the same value for facetfield. I 
tried this with clustered and single SOLR node.

Expected

Actual

"sc_uf_facetfield": {
"buckets": [
{
"val": "facet value",
   "count": 1
}
]
}

"sc_uf_facetfield": {
"buckets": [
{
"val": "facet value",
"count": 2
}
]
}


Can you please let me know whether this is a known SOLR bug or my SOLR query is 
wrong?


Thanks,
Vinoth

Use "relatedness(...)" faceting function as a replacement for MoreLikeThis?

[Scott Bouma]

Hi all!  First question to the group so hopefully this is the right way to
ask a question :)

I watched one of Trey Grainger's talks in which he said that the new
relatedness metric is an improvement on MoreLikeThis (MLT).  I would like
to use it to analyze a set of text files from different publishers
(multiple files per publisher) in order to determine relatedness between
publishers.  I managed to get it to work with MLT by running the query
below (I post-processed the results to sum MLT scores between each
publisher pair).  But I can't figure out how to do a similar analysis using
facets and the relatedness(...) function.

http://localhost:8983/solr/plans/select?fl=score%2Cpublisher_url%2Cid=data.devotional=true=*%3A*

Any suggestions would be appreciated!  Thanks,
-Scott

Re: Ignore faceting for particular fields in solr using Solrconfig.xml

[Bernd Fehling]

Have a look at "invariants" for your requestHandler in solrconfig.xml.
It might be an option for you.

Regards
Bernd


Am 22.05.19 um 22:23 schrieb RaviTeja:

Hello Solr Expert,

How are you?

Am trying to ignore faceting for some of the fields. Can you please help me
out to ignore faceting using solrconfig.xml.
I tried but I can ignore faceting all the fields that useless. I'm trying
to ignore some specific fields.

Really Appreciate your help for the response!

Regards,
Ravi

Re: Ignore faceting for particular fields in solr using Solrconfig.xml

[Erick Erickson]

Just don’t ask for them. Or you saying that users can specify arbitrary fields 
to facet on and you want to prevent certain fields from being possible?

No, there’s no good way to do that in solrconfig.xml. You could write a query 
component that stripped out certain fields from the facet.field parameter.

Likely the easiest would be to do that in the application I assume you have 
between Solr and your users.

Best,
Erick

> On May 22, 2019, at 1:23 PM, RaviTeja  wrote:
> 
> Hello Solr Expert,
> 
> How are you?
> 
> Am trying to ignore faceting for some of the fields. Can you please help me
> out to ignore faceting using solrconfig.xml.
> I tried but I can ignore faceting all the fields that useless. I'm trying
> to ignore some specific fields.
> 
> Really Appreciate your help for the response!
> 
> Regards,
> Ravi

Ignore faceting for particular fields in solr using Solrconfig.xml

[RaviTeja]

Hello Solr Expert,

How are you?

Am trying to ignore faceting for some of the fields. Can you please help me
out to ignore faceting using solrconfig.xml.
I tried but I can ignore faceting all the fields that useless. I'm trying
to ignore some specific fields.

Really Appreciate your help for the response!

Regards,
Ravi

Re: Slow faceting performance on a docValues field

[gulats]

maybe quite late to the party but for the benefit of future readers,
experimentation with facet.range.method might be helpful (for solr versions
6 and above) as it allows us to use docValues as well for range faceting



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: genre_str field in Solr's official tutorial on faceting

[Jan Høydahl]

genre_str is a pure string copyField created from the original genre field, but 
it is not returned in results because it has stored=false and it would return 
exactly the same value as genre. 

The _str copyFileds are created automatically when using schemaless mode.

You can inspect the scema in Schema browser in the admin UI.

Jan Høydahl

> 27. apr. 2019 kl. 21:06 skrev Sharmadha :
> 
> Following
> https://lucene.apache.org/solr/guide/7_7/solr-tutorial.html#field-facets , I
> see in the documents , there is only genre field. In the example for
> faceting , which filed do they refer by "genre_str" ? Are genre and
> genre_str the same ? But I see different result when I facet on them. 
> 
> Kinldy help If I am missing something basic.
> 
> 
> 
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

genre_str field in Solr's official tutorial on faceting

[Sharmadha]

Following
https://lucene.apache.org/solr/guide/7_7/solr-tutorial.html#field-facets , I
see in the documents , there is only genre field. In the example for
faceting , which filed do they refer by "genre_str" ? Are genre and
genre_str the same ? But I see different result when I facet on them. 

Kinldy help If I am missing something basic.



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Solr 7.7 - group faceting errors

[Jay Potharaju]

Hi,
I am running into a bug when doing group faceting. This is the same error I
ran into when upgrading from 5.5 to 6.x.
http://lucene.472066.n3.nabble.com/solr-6-6-3-intermittent-group-faceting-errors-td4385692.html#a4385865
I had bypassed the error in solr 6.x by turning off docvalues. But in solr
7.7 it wont let me do faceting without enabling docvalues.
Anyone else ran into this issue?

Sample Document
{ "id":"123:34!1:1",
"product_id":1,
"sku_id":1
},
{
"id":"123:34!2:1",
"category_id": [1,2]
"product_id":2,
"sku_id":1
}
solr 7.7 schema




solr 6..6 schema





 2019-03-29 20:14:30.188 ERROR (qtp2051853139-14) [c:collection1
s:shard2 r:core_node4 x:collection1_shard2_replica_n2]
o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException:
Exception during facet.field: category_id
at 
org.apache.solr.request.SimpleFacets.lambda$getFacetFieldCounts$0(SimpleFacets.java:832)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.request.SimpleFacets$3.execute(SimpleFacets.java:771)
at 
org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:841)
at 
org.apache.solr.handler.component.FacetComponent.getFacetCounts(FacetComponent.java:329)
at 
org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:273)
at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:298)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2551)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:710)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:516)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:395)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:341)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)
at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588)
at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557)
at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)
at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at 
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:502)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
at 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
at 
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
at 
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
at 
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at 
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
 

Re: Faceting filter tagging doesn't work in case where 0 matches are found

[Mikhail Khludnev]

I've consulted regarding this case. This is not an issue, you may bring
facet back adding not yet documented property processEmpty:true

On Mon, Feb 18, 2019 at 10:42 AM Mikhail Khludnev  wrote:

> Hello,
> I'm not sure but it sounds like an issue, would you mind to raise one at
> https://issues.apache.org/jira/projects/SOLR/ ?
>
> On Sun, Feb 17, 2019 at 6:57 PM Arvydas Silanskas <
> nma.arvydas.silans...@gmail.com> wrote:
>
>> Good evening,
>>
>> I am using facet json api to query aggregation data, and I don't care
>> about
>> the returned documents themselves. One of the use cases I want to employ
>> is
>> tagging filter queries for fields, and then exclude those filters when
>> faceting. My problem is, however, that in those cases where the filter has
>> 0 matches, the facets aren't calculated at all.
>>
>> I'm using dataset I found at
>> https://www.raspberry.nl/2010/12/29/solr-test-dataset/ . To illustrate --
>> this is an an example when filter doesn't filter out everything (working
>> as
>> expected):
>>
>> Request:
>> {
>>   "query": "*:*",
>>   "facet": {
>> "latitude_f": {
>>   "type": "range",
>>   "start": -90,
>>   "facet": {
>> "population": "sum(population_i)"
>>   },
>>   "domain": {
>> "excludeTags": "latitude_f"
>>   },
>>   "gap": 10,
>>   "end": -70,
>>   "field": "latitude_f"
>> }
>>   },
>>   "limit": 0,
>>   "filter": [
>> "{!tag=latitude_f}latitude_f:[-80.0 TO -70.0]"
>>   ]
>> }
>>
>> Response:
>>
>> {
>>   "facets": {
>> "count": 1,
>> "latitude_f": {
>>   "buckets": [
>> {
>>   "val": -90,
>>   "count": 0
>> },
>> {
>>   "val": -80,
>>   "count": 1,
>>   "population": 1258
>> }
>>   ]
>> }
>>   }
>> }
>>
>>
>> Example when filter filters everything out:
>>
>> Request is the same, except the filter field value is
>>
>>   "filter": [
>> "{!tag=latitude_f}latitude_f:[-90.0 TO -80.0]"
>>   ]
>>
>> and response is
>>
>>  "facets":{
>> "count":0}
>>
>> . I'm returned no facets whatsoever. However I'd expect the response to be
>> the same as and for the first request, since the only one filter is used,
>> and is excluded in faceting.
>>
>> Is this a bug? What are the workarounds for such problem?
>>
>
>
> --
> Sincerely yours
> Mikhail Khludnev
>


-- 
Sincerely yours
Mikhail Khludnev

Re: Faceting filter tagging doesn't work in case where 0 matches are found

[Zheng Lin Edwin Yeo]

Hi,

Which version of Solr are you using when you face this problem?

Regards,
Edwin

On Mon, 18 Feb 2019 at 15:43, Mikhail Khludnev  wrote:

> Hello,
> I'm not sure but it sounds like an issue, would you mind to raise one at
> https://issues.apache.org/jira/projects/SOLR/ ?
>
> On Sun, Feb 17, 2019 at 6:57 PM Arvydas Silanskas <
> nma.arvydas.silans...@gmail.com> wrote:
>
> > Good evening,
> >
> > I am using facet json api to query aggregation data, and I don't care
> about
> > the returned documents themselves. One of the use cases I want to employ
> is
> > tagging filter queries for fields, and then exclude those filters when
> > faceting. My problem is, however, that in those cases where the filter
> has
> > 0 matches, the facets aren't calculated at all.
> >
> > I'm using dataset I found at
> > https://www.raspberry.nl/2010/12/29/solr-test-dataset/ . To illustrate
> --
> > this is an an example when filter doesn't filter out everything (working
> as
> > expected):
> >
> > Request:
> > {
> >   "query": "*:*",
> >   "facet": {
> > "latitude_f": {
> >   "type": "range",
> >   "start": -90,
> >   "facet": {
> > "population": "sum(population_i)"
> >   },
> >   "domain": {
> > "excludeTags": "latitude_f"
> >   },
> >   "gap": 10,
> >   "end": -70,
> >   "field": "latitude_f"
> > }
> >   },
> >   "limit": 0,
> >   "filter": [
> > "{!tag=latitude_f}latitude_f:[-80.0 TO -70.0]"
> >   ]
> > }
> >
> > Response:
> >
> > {
> >   "facets": {
> > "count": 1,
> > "latitude_f": {
> >   "buckets": [
> > {
> >   "val": -90,
> >   "count": 0
> > },
> >     {
> >   "val": -80,
> >   "count": 1,
> >   "population": 1258
> > }
> >   ]
> > }
> >   }
> > }
> >
> >
> > Example when filter filters everything out:
> >
> > Request is the same, except the filter field value is
> >
> >   "filter": [
> > "{!tag=latitude_f}latitude_f:[-90.0 TO -80.0]"
> >   ]
> >
> > and response is
> >
> >  "facets":{
> > "count":0}
> >
> > . I'm returned no facets whatsoever. However I'd expect the response to
> be
> > the same as and for the first request, since the only one filter is used,
> > and is excluded in faceting.
> >
> > Is this a bug? What are the workarounds for such problem?
> >
>
>
> --
> Sincerely yours
> Mikhail Khludnev
>

Re: Faceting filter tagging doesn't work in case where 0 matches are found

[Mikhail Khludnev]

Hello,
I'm not sure but it sounds like an issue, would you mind to raise one at
https://issues.apache.org/jira/projects/SOLR/ ?

On Sun, Feb 17, 2019 at 6:57 PM Arvydas Silanskas <
nma.arvydas.silans...@gmail.com> wrote:

> Good evening,
>
> I am using facet json api to query aggregation data, and I don't care about
> the returned documents themselves. One of the use cases I want to employ is
> tagging filter queries for fields, and then exclude those filters when
> faceting. My problem is, however, that in those cases where the filter has
> 0 matches, the facets aren't calculated at all.
>
> I'm using dataset I found at
> https://www.raspberry.nl/2010/12/29/solr-test-dataset/ . To illustrate --
> this is an an example when filter doesn't filter out everything (working as
> expected):
>
> Request:
> {
>   "query": "*:*",
>   "facet": {
> "latitude_f": {
>   "type": "range",
>   "start": -90,
>   "facet": {
> "population": "sum(population_i)"
>   },
>   "domain": {
> "excludeTags": "latitude_f"
>   },
>   "gap": 10,
>   "end": -70,
>   "field": "latitude_f"
> }
>   },
>   "limit": 0,
>   "filter": [
> "{!tag=latitude_f}latitude_f:[-80.0 TO -70.0]"
>   ]
> }
>
> Response:
>
> {
>   "facets": {
> "count": 1,
> "latitude_f": {
>   "buckets": [
> {
>   "val": -90,
>   "count": 0
> },
> {
>   "val": -80,
>   "count": 1,
>   "population": 1258
>     }
>   ]
> }
>   }
> }
>
>
> Example when filter filters everything out:
>
> Request is the same, except the filter field value is
>
>   "filter": [
> "{!tag=latitude_f}latitude_f:[-90.0 TO -80.0]"
>   ]
>
> and response is
>
>  "facets":{
> "count":0}
>
> . I'm returned no facets whatsoever. However I'd expect the response to be
> the same as and for the first request, since the only one filter is used,
> and is excluded in faceting.
>
> Is this a bug? What are the workarounds for such problem?
>


-- 
Sincerely yours
Mikhail Khludnev

Faceting filter tagging doesn't work in case where 0 matches are found

[Arvydas Silanskas]

Good evening,

I am using facet json api to query aggregation data, and I don't care about
the returned documents themselves. One of the use cases I want to employ is
tagging filter queries for fields, and then exclude those filters when
faceting. My problem is, however, that in those cases where the filter has
0 matches, the facets aren't calculated at all.

I'm using dataset I found at
https://www.raspberry.nl/2010/12/29/solr-test-dataset/ . To illustrate --
this is an an example when filter doesn't filter out everything (working as
expected):

Request:
{
  "query": "*:*",
  "facet": {
"latitude_f": {
  "type": "range",
  "start": -90,
  "facet": {
"population": "sum(population_i)"
  },
  "domain": {
"excludeTags": "latitude_f"
  },
  "gap": 10,
  "end": -70,
  "field": "latitude_f"
}
  },
  "limit": 0,
  "filter": [
"{!tag=latitude_f}latitude_f:[-80.0 TO -70.0]"
  ]
}

Response:

{
  "facets": {
"count": 1,
"latitude_f": {
  "buckets": [
{
  "val": -90,
  "count": 0
},
{
  "val": -80,
  "count": 1,
  "population": 1258
}
  ]
}
  }
}


Example when filter filters everything out:

Request is the same, except the filter field value is

  "filter": [
"{!tag=latitude_f}latitude_f:[-90.0 TO -80.0]"
  ]

and response is

 "facets":{
"count":0}

. I'm returned no facets whatsoever. However I'd expect the response to be
the same as and for the first request, since the only one filter is used,
and is excluded in faceting.

Is this a bug? What are the workarounds for such problem?

Re: Faceting with a multi valued field

[Shawn Heisey]

On 9/25/2018 2:14 PM, Hanjan, Harinder wrote:

Hello!


When starting a new topic on the mailing list, do not reply to an 
existing message.  Your thread is buried within a thread originally 
titled "Extracting top level URL when indexing document".


https://home.apache.org/~hossman/#threadhijack


Notice that the Communities facet has 2 non zero results. I understand this is 
because I'm using fq to get only documents which contain BANFF TRAIL but those 
documents also contain PARKDALE.


Facets return information for what the document that match the query 
contain.  ALL of the information.  The query that returned those matches 
is not examined at all when calculating facets, only the *results* of 
the query are examined.  I don't think there's any way you can exclude 
the information that you want to exclude, other than removing it from 
the documents entirely.  I would imagine that the PARKDALE information 
is required in those documents for other purposes and probably can't be 
removed.


Thanks,
Shawn

RE: [EXT] Re: Faceting with a multi valued field

[Hanjan, Harinder]

I control everything except the data that's being indexed. So I can manipulate 
the Solr query as needed.

I tried the facet.prefix option and initial testing shows promise. 
q=*:*=on=Communities=BANFF+TRAIL+-+BNF

Thanks much! 


-Original Message-
From: Alexandre Rafalovitch [mailto:arafa...@gmail.com] 
Sent: Tuesday, September 25, 2018 3:14 PM
To: solr-user
Subject: [EXT] Re: Faceting with a multi valued field

What specifically do you control? Just keyword (and "Communities:"
part is locked?) or anything after q= or anything that allows multiple 
variables?

Because if you could isolate search value, you could use for example 
facet.prefix, set in solrconfig as a default parameter and populated from the 
same variable as the Communities search.

You may also want to set facet.mincount=1 in solrconfig.xml to avoid 0-value 
facets in general:
https://urldefense.proofpoint.com/v2/url?u=https-3A__lucene.apache.org_solr_guide_7-5F4_faceting.html=DwIBaQ=jdm1Hby_BzoqwoYzPsUCHSCnNps9LuidNkyKDuvdq3M=N30IrhmaeKKhVHu13d-HO9gO9CysWnvGGoKrSNEuM3U=xAdIgtTdaZYLG3jsYsLQqWtQBb9-cHsyG58r_mvTm-E=RgNvfB_bRwAfe9NpY1HedFlSHUNY0QbZ4VCXTzduTMo=

Regards,
   Alex.


On 25 September 2018 at 16:50, John Blythe  wrote:
> you can update your filter query to be a facet query, this will apply 
> the query to the resulting facet set instead of the Communities field itself.
>
> --
> John Blythe
>
>
> On Tue, Sep 25, 2018 at 4:15 PM Hanjan, Harinder 
> 
> wrote:
>
>> Hello!
>>
>> I am doing faceting on a field which has multiple values and it's 
>> yielding expected but undesireable results. I need different 
>> behaviour but not sure how to formulate a query for it. Here is my current 
>> setup.
>>
>> = Data Set =
>>   {
>> "Communities":["BANFF TRAIL - BNF", "PARKDALE - PKD"], "Document 
>> Type":"Engagement - What We Heard Report", "Navigation":"Livelink", 
>> "SolrId":"https://urldefense.proofpoint.com/v2/url?u=http-3A__thesimpsons.com_one=DwIBaQ=jdm1Hby_BzoqwoYzPsUCHSCnNps9LuidNkyKDuvdq3M=N30IrhmaeKKhVHu13d-HO9gO9CysWnvGGoKrSNEuM3U=xAdIgtTdaZYLG3jsYsLQqWtQBb9-cHsyG58r_mvTm-E=-ZCoMFGNAEILlQOvY1Stra9dCF-rM48tZSTT3QJcOA0=;
>>   }
>>   {
>> "Communities":["BANFF TRAIL - BNF", "PARKDALE - PKD"], "Document 
>> Type":"Engagement - What We Heard Report", "Navigation":"Livelink", 
>> "Id":"https://urldefense.proofpoint.com/v2/url?u=http-3A__thesimpsons.com_two=DwIBaQ=jdm1Hby_BzoqwoYzPsUCHSCnNps9LuidNkyKDuvdq3M=N30IrhmaeKKhVHu13d-HO9gO9CysWnvGGoKrSNEuM3U=xAdIgtTdaZYLG3jsYsLQqWtQBb9-cHsyG58r_mvTm-E=_JPFUX0e0zqyJWHQzWH815ThZAsdGu5TwDSkXBIL23Q=;
>>   }
>>   {
>> "Communities":["SUNALTA - SNA"],
>> "Document Type":"Engagement - What We Heard Report", 
>> "Navigation":"Livelink", 
>> "Id":"https://urldefense.proofpoint.com/v2/url?u=http-3A__thesimpsons.com_three=DwIBaQ=jdm1Hby_BzoqwoYzPsUCHSCnNps9LuidNkyKDuvdq3M=N30IrhmaeKKhVHu13d-HO9gO9CysWnvGGoKrSNEuM3U=xAdIgtTdaZYLG3jsYsLQqWtQBb9-cHsyG58r_mvTm-E=scFc0GYxSyRaAiAmu4M3AvYNiMsgqffG1Jmko76YjH8=;
>>   }
>>
>> = Query I run now =
>>
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__localhost-3A8984_
>> solr_everything_select-3Fq-3D-2A-3A-2A-26facet-3Don-26facet.field-3DC
>> ommunities-26fq-3DCommunities-3A-2522BANFF=DwIBaQ=jdm1Hby_BzoqwoY
>> zPsUCHSCnNps9LuidNkyKDuvdq3M=N30IrhmaeKKhVHu13d-HO9gO9CysWnvGGoKrSN
>> EuM3U=xAdIgtTdaZYLG3jsYsLQqWtQBb9-cHsyG58r_mvTm-E=G7NJKKdDNh0wP5l
>> sjrSQnmbT77hUTSgx2giYBuQFdEI=
>> TRAIL - BNF"
>>
>>
>> = Results I get now =
>> {
>>   ...
>>   "facet_counts":{
>> "facet_queries":{},
>> "facet_fields":{
>>   "Communities":[
>> "BANFF TRAIL - BNF",2,
>> "PARKDALE - PKD",2,
>> "SUNALTA - SNA",0]},
>>...
>>
>> Notice that the Communities facet has 2 non zero results. I 
>> understand this is because I'm using fq to get only documents which 
>> contain BANFF TRAIL but those documents also contain PARKDALE.
>>
>> Now, I am using facets to drive navigation on my page. The business 
>> case is that user can select a community to get documents pertaining 
>> to that specific community only. This works with the query I have 
>> above. However, the facets results also contain other communities 
>> which then get displayed to the user. For example, with the query 
>> above, user will see both BANFF TRAIL and

RE: [EXT] Re: Faceting with a multi valued field

[Hanjan, Harinder]

John,

I just want to make sure I understand correctly. Replace, fq with facet.query?

So then the resultant query goes from:
q=*:*=on=Communities=Communities:"BANFF TRAIL - BNF"

to:
q=*:*=on=Communities="BANFF TRAIL - BNF"


If that's correct, then this does not resolve the issue. I still get 2 values 
under Communities facet.

Harinder

-Original Message-
From: John Blythe [mailto:johnbly...@gmail.com] 
Sent: Tuesday, September 25, 2018 2:50 PM
To: solr-user@lucene.apache.org
Subject: [EXT] Re: Faceting with a multi valued field

you can update your filter query to be a facet query, this will apply the query 
to the resulting facet set instead of the Communities field itself.

--
John Blythe


On Tue, Sep 25, 2018 at 4:15 PM Hanjan, Harinder 
wrote:

> Hello!
>
> I am doing faceting on a field which has multiple values and it's 
> yielding expected but undesireable results. I need different behaviour 
> but not sure how to formulate a query for it. Here is my current setup.
>
> = Data Set =
>   {
> "Communities":["BANFF TRAIL - BNF", "PARKDALE - PKD"], "Document 
> Type":"Engagement - What We Heard Report", "Navigation":"Livelink", 
> "SolrId":"https://urldefense.proofpoint.com/v2/url?u=http-3A__thesimpsons.com_one=DwIBaQ=jdm1Hby_BzoqwoYzPsUCHSCnNps9LuidNkyKDuvdq3M=N30IrhmaeKKhVHu13d-HO9gO9CysWnvGGoKrSNEuM3U=PX7TJqsA8tYgbN7HmkGd0GNzotXPc3hcoc9xRvmOiXI=GMTgF731T72VIryx_v7VD5f_oBlbrzXYAB1UEBQMOOc=;
>   }
>   {
> "Communities":["BANFF TRAIL - BNF", "PARKDALE - PKD"], "Document 
> Type":"Engagement - What We Heard Report", "Navigation":"Livelink", 
> "Id":"https://urldefense.proofpoint.com/v2/url?u=http-3A__thesimpsons.com_two=DwIBaQ=jdm1Hby_BzoqwoYzPsUCHSCnNps9LuidNkyKDuvdq3M=N30IrhmaeKKhVHu13d-HO9gO9CysWnvGGoKrSNEuM3U=PX7TJqsA8tYgbN7HmkGd0GNzotXPc3hcoc9xRvmOiXI=FN6T49z8wjc_mRdXnVHgcdZBcZB6O_InSyUzxaxxiM0=;
>   }
>   {
> "Communities":["SUNALTA - SNA"],
> "Document Type":"Engagement - What We Heard Report", 
> "Navigation":"Livelink", 
> "Id":"https://urldefense.proofpoint.com/v2/url?u=http-3A__thesimpsons.com_three=DwIBaQ=jdm1Hby_BzoqwoYzPsUCHSCnNps9LuidNkyKDuvdq3M=N30IrhmaeKKhVHu13d-HO9gO9CysWnvGGoKrSNEuM3U=PX7TJqsA8tYgbN7HmkGd0GNzotXPc3hcoc9xRvmOiXI=HEJFyAhHIn5T-riqVVMR011KXAn38lZUDyRQ-ljC-qA=;
>   }
>
> = Query I run now =
>
> https://urldefense.proofpoint.com/v2/url?u=http-3A__localhost-3A8984_s
> olr_everything_select-3Fq-3D-2A-3A-2A-26facet-3Don-26facet.field-3DCom
> munities-26fq-3DCommunities-3A-2522BANFF=DwIBaQ=jdm1Hby_BzoqwoYzPs
> UCHSCnNps9LuidNkyKDuvdq3M=N30IrhmaeKKhVHu13d-HO9gO9CysWnvGGoKrSNEuM3
> U=PX7TJqsA8tYgbN7HmkGd0GNzotXPc3hcoc9xRvmOiXI=Cx6EubqN_-ocYrZA6jsJ
> TGzodPqUPVu78eY1iMB_0L8=
> TRAIL - BNF"
>
>
> = Results I get now =
> {
>   ...
>   "facet_counts":{
> "facet_queries":{},
> "facet_fields":{
>   "Communities":[
> "BANFF TRAIL - BNF",2,
> "PARKDALE - PKD",2,
> "SUNALTA - SNA",0]},
>...
>
> Notice that the Communities facet has 2 non zero results. I understand 
> this is because I'm using fq to get only documents which contain BANFF 
> TRAIL but those documents also contain PARKDALE.
>
> Now, I am using facets to drive navigation on my page. The business 
> case is that user can select a community to get documents pertaining 
> to that specific community only. This works with the query I have 
> above. However, the facets results also contain other communities 
> which then get displayed to the user. For example, with the query 
> above, user will see both BANFF TRAIL and PARKDALE as selected values 
> even though user only selected BANFF TRAIL. It's worthwhile noting 
> that I have no control over the data being sent to Solr and can't change it.
>
> How can I formulate a query to ensure that when user selects BANFF 
> TRAIL, only BANFF TRAIL is returned under Solr facets?
>
> Thanks!
> Harinder
>
> 
> NOTICE -
> This communication is intended ONLY for the use of the person or 
> entity named above and may contain information that is confidential or 
> legally privileged. If you are not the intended recipient named above 
> or a person responsible for delivering messages or communications to 
> the intended recipient, YOU ARE HEREBY NOTIFIED that any use, 
> distribution, or copying of this communication or any of the 
> information contained in it is strictly prohibited. If you have 
> received this communication in error, please notify us immediately by 
> telephone and then destroy or delete this communication, or return it 
> to us by mail if requested by us. The City of Calgary thanks you for your 
> attention and co-operation.
>

Re: Faceting with a multi valued field

[Alexandre Rafalovitch]

What specifically do you control? Just keyword (and "Communities:"
part is locked?) or anything after q= or anything that allows multiple
variables?

Because if you could isolate search value, you could use for example
facet.prefix, set in solrconfig as a default parameter and populated
from the same variable as the Communities search.

You may also want to set facet.mincount=1 in solrconfig.xml to avoid
0-value facets in general:
https://lucene.apache.org/solr/guide/7_4/faceting.html

Regards,
   Alex.


On 25 September 2018 at 16:50, John Blythe  wrote:
> you can update your filter query to be a facet query, this will apply the
> query to the resulting facet set instead of the Communities field itself.
>
> --
> John Blythe
>
>
> On Tue, Sep 25, 2018 at 4:15 PM Hanjan, Harinder 
> wrote:
>
>> Hello!
>>
>> I am doing faceting on a field which has multiple values and it's yielding
>> expected but undesireable results. I need different behaviour but not sure
>> how to formulate a query for it. Here is my current setup.
>>
>> = Data Set =
>>   {
>> "Communities":["BANFF TRAIL - BNF", "PARKDALE - PKD"],
>> "Document Type":"Engagement - What We Heard Report",
>> "Navigation":"Livelink",
>> "SolrId":"http://thesimpsons.com/one;
>>   }
>>   {
>> "Communities":["BANFF TRAIL - BNF", "PARKDALE - PKD"],
>> "Document Type":"Engagement - What We Heard Report",
>> "Navigation":"Livelink",
>> "Id":"http://thesimpsons.com/two;
>>   }
>>   {
>> "Communities":["SUNALTA - SNA"],
>> "Document Type":"Engagement - What We Heard Report",
>> "Navigation":"Livelink",
>> "Id":"http://thesimpsons.com/three;
>>   }
>>
>> = Query I run now =
>>
>> http://localhost:8984/solr/everything/select?q=*:*=on=Communities=Communities:"BANFF
>> TRAIL - BNF"
>>
>>
>> = Results I get now =
>> {
>>   ...
>>   "facet_counts":{
>> "facet_queries":{},
>> "facet_fields":{
>>   "Communities":[
>> "BANFF TRAIL - BNF",2,
>> "PARKDALE - PKD",2,
>> "SUNALTA - SNA",0]},
>>...
>>
>> Notice that the Communities facet has 2 non zero results. I understand
>> this is because I'm using fq to get only documents which contain BANFF
>> TRAIL but those documents also contain PARKDALE.
>>
>> Now, I am using facets to drive navigation on my page. The business case
>> is that user can select a community to get documents pertaining to that
>> specific community only. This works with the query I have above. However,
>> the facets results also contain other communities which then get displayed
>> to the user. For example, with the query above, user will see both BANFF
>> TRAIL and PARKDALE as selected values even though user only selected BANFF
>> TRAIL. It's worthwhile noting that I have no control over the data being
>> sent to Solr and can't change it.
>>
>> How can I formulate a query to ensure that when user selects BANFF TRAIL,
>> only BANFF TRAIL is returned under Solr facets?
>>
>> Thanks!
>> Harinder
>>
>> 
>> NOTICE -
>> This communication is intended ONLY for the use of the person or entity
>> named above and may contain information that is confidential or legally
>> privileged. If you are not the intended recipient named above or a person
>> responsible for delivering messages or communications to the intended
>> recipient, YOU ARE HEREBY NOTIFIED that any use, distribution, or copying
>> of this communication or any of the information contained in it is strictly
>> prohibited. If you have received this communication in error, please notify
>> us immediately by telephone and then destroy or delete this communication,
>> or return it to us by mail if requested by us. The City of Calgary thanks
>> you for your attention and co-operation.
>>

Re: Faceting with a multi valued field

[John Blythe]

you can update your filter query to be a facet query, this will apply the
query to the resulting facet set instead of the Communities field itself.

--
John Blythe


On Tue, Sep 25, 2018 at 4:15 PM Hanjan, Harinder 
wrote:

> Hello!
>
> I am doing faceting on a field which has multiple values and it's yielding
> expected but undesireable results. I need different behaviour but not sure
> how to formulate a query for it. Here is my current setup.
>
> = Data Set =
>   {
> "Communities":["BANFF TRAIL - BNF", "PARKDALE - PKD"],
> "Document Type":"Engagement - What We Heard Report",
> "Navigation":"Livelink",
> "SolrId":"http://thesimpsons.com/one;
>   }
>   {
> "Communities":["BANFF TRAIL - BNF", "PARKDALE - PKD"],
> "Document Type":"Engagement - What We Heard Report",
> "Navigation":"Livelink",
> "Id":"http://thesimpsons.com/two;
>   }
>   {
> "Communities":["SUNALTA - SNA"],
> "Document Type":"Engagement - What We Heard Report",
> "Navigation":"Livelink",
> "Id":"http://thesimpsons.com/three;
>   }
>
> = Query I run now =
>
> http://localhost:8984/solr/everything/select?q=*:*=on=Communities=Communities:"BANFF
> TRAIL - BNF"
>
>
> = Results I get now =
> {
>   ...
>   "facet_counts":{
> "facet_queries":{},
> "facet_fields":{
>   "Communities":[
> "BANFF TRAIL - BNF",2,
> "PARKDALE - PKD",2,
> "SUNALTA - SNA",0]},
>...
>
> Notice that the Communities facet has 2 non zero results. I understand
> this is because I'm using fq to get only documents which contain BANFF
> TRAIL but those documents also contain PARKDALE.
>
> Now, I am using facets to drive navigation on my page. The business case
> is that user can select a community to get documents pertaining to that
> specific community only. This works with the query I have above. However,
> the facets results also contain other communities which then get displayed
> to the user. For example, with the query above, user will see both BANFF
> TRAIL and PARKDALE as selected values even though user only selected BANFF
> TRAIL. It's worthwhile noting that I have no control over the data being
> sent to Solr and can't change it.
>
> How can I formulate a query to ensure that when user selects BANFF TRAIL,
> only BANFF TRAIL is returned under Solr facets?
>
> Thanks!
> Harinder
>
> 
> NOTICE -
> This communication is intended ONLY for the use of the person or entity
> named above and may contain information that is confidential or legally
> privileged. If you are not the intended recipient named above or a person
> responsible for delivering messages or communications to the intended
> recipient, YOU ARE HEREBY NOTIFIED that any use, distribution, or copying
> of this communication or any of the information contained in it is strictly
> prohibited. If you have received this communication in error, please notify
> us immediately by telephone and then destroy or delete this communication,
> or return it to us by mail if requested by us. The City of Calgary thanks
> you for your attention and co-operation.
>

Faceting with a multi valued field

[Hanjan, Harinder]

Hello!

I am doing faceting on a field which has multiple values and it's yielding 
expected but undesireable results. I need different behaviour but not sure how 
to formulate a query for it. Here is my current setup.

= Data Set =
  {
"Communities":["BANFF TRAIL - BNF", "PARKDALE - PKD"],
"Document Type":"Engagement - What We Heard Report",
"Navigation":"Livelink",
"SolrId":"http://thesimpsons.com/one;
  }
  {
"Communities":["BANFF TRAIL - BNF", "PARKDALE - PKD"],
"Document Type":"Engagement - What We Heard Report",
"Navigation":"Livelink",
"Id":"http://thesimpsons.com/two;
  }
  {
"Communities":["SUNALTA - SNA"],
"Document Type":"Engagement - What We Heard Report",
"Navigation":"Livelink",
"Id":"http://thesimpsons.com/three;
  }

= Query I run now =
http://localhost:8984/solr/everything/select?q=*:*=on=Communities=Communities:"BANFF
 TRAIL - BNF"


= Results I get now =
{
  ...
  "facet_counts":{
"facet_queries":{},
"facet_fields":{
  "Communities":[
"BANFF TRAIL - BNF",2,
"PARKDALE - PKD",2,
"SUNALTA - SNA",0]},
   ...

Notice that the Communities facet has 2 non zero results. I understand this is 
because I'm using fq to get only documents which contain BANFF TRAIL but those 
documents also contain PARKDALE.

Now, I am using facets to drive navigation on my page. The business case is 
that user can select a community to get documents pertaining to that specific 
community only. This works with the query I have above. However, the facets 
results also contain other communities which then get displayed to the user. 
For example, with the query above, user will see both BANFF TRAIL and PARKDALE 
as selected values even though user only selected BANFF TRAIL. It's worthwhile 
noting that I have no control over the data being sent to Solr and can't change 
it.

How can I formulate a query to ensure that when user selects BANFF TRAIL, only 
BANFF TRAIL is returned under Solr facets?

Thanks!
Harinder


NOTICE -
This communication is intended ONLY for the use of the person or entity named 
above and may contain information that is confidential or legally privileged. 
If you are not the intended recipient named above or a person responsible for 
delivering messages or communications to the intended recipient, YOU ARE HEREBY 
NOTIFIED that any use, distribution, or copying of this communication or any of 
the information contained in it is strictly prohibited. If you have received 
this communication in error, please notify us immediately by telephone and then 
destroy or delete this communication, or return it to us by mail if requested 
by us. The City of Calgary thanks you for your attention and co-operation.

Re: Faceting with EnumFieldType in 7.1

[Walter Underwood]

Yes.

Consider search for a bug database with severity levels in an enum field. A 
facet on severity would be a normal feature for that search.

I would call this a bug.

wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/  (my blog)

> On Sep 20, 2018, at 8:41 AM, Peter Tyrrell  wrote:
> 
> I have to assume this is a bug. (EnumFieldType does not return facet values.) 
> I didn't want to create a ticket before floating the issue on this mailing 
> list, but absent any further guidance or discussion...
> 
> So one last shot. Should a field of EnumFieldType be useable as a facet field?
> 
> Thanks,
> 
> Peter
> 
> Peter Tyrrell, MLIS
> Lead Developer at Andornot
> 1-866-266-2525 x706 / ptyrr...@andornot.com
> 
> -Original Message-
> From: Peter Tyrrell 
> Sent: September 14, 2018 3:04 PM
> To: solr-user@lucene.apache.org
> Subject: RE: Faceting with EnumFieldType in 7.1
> 
> Yes.
> 
> Peter Tyrrell, MLIS
> Lead Developer at Andornot
> 1-866-266-2525 x706 / ptyrr...@andornot.com
> 
> -Original Message-
> From: Zheng Lin Edwin Yeo 
> Sent: September 13, 2018 8:15 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Faceting with EnumFieldType in 7.1
> 
> Was the document re-indexed in Solr 7.1?
> 
> Regards,
> Edwin
> 
> On Wed, 12 Sep 2018 at 23:38, Peter Tyrrell  wrote:
> 
>> I updated an older Solr 4.10 core to Solr 7.1 recently. In so doing, I 
>> took an old 'gradeLevel_enum' field of type EnumField and made it an 
>> EnumFieldType, since the former has been deprecated. The old core was 
>> able to facet on gradeLevel_enum, but the new 7.1 core just returns no 
>> facet values whatsoever for that field. Both cores return 
>> gradeLevel_enum values ok when fl=gradeLevel_enum.
>> 
>> In the schema, gradeLevel_enum is defined dynamically:
>> 
>> > multiValued="true" />
>> > enumsConfig="enumsConfig.xml" enumName="gradeLevels">
>> 
>> This simple query fails to return any facet values in 7.1, but does 
>> facet in 4.10:
>> 
>> 
>> http://localhost:8983/solr/core1/select?facet.field=gradeLevel_enum
>> cet=on=id,gradeLevel_enum=*:*=json
>> 
>> Thanks for any insight.
>> 
>> Peter Tyrrell, MLIS
>> Lead Developer at Andornot
>> 1-866-266-2525 x706 /
>> ptyrr...@andornot.com<mailto:ptyrr...@andornot.com>
>> 
>> 

RE: Faceting with EnumFieldType in 7.1

[Peter Tyrrell]

I have to assume this is a bug. (EnumFieldType does not return facet values.) I 
didn't want to create a ticket before floating the issue on this mailing list, 
but absent any further guidance or discussion...

So one last shot. Should a field of EnumFieldType be useable as a facet field?

Thanks,

Peter

Peter Tyrrell, MLIS
Lead Developer at Andornot
1-866-266-2525 x706 / ptyrr...@andornot.com

-Original Message-
From: Peter Tyrrell 
Sent: September 14, 2018 3:04 PM
To: solr-user@lucene.apache.org
Subject: RE: Faceting with EnumFieldType in 7.1

Yes.

Peter Tyrrell, MLIS
Lead Developer at Andornot
1-866-266-2525 x706 / ptyrr...@andornot.com

-Original Message-
From: Zheng Lin Edwin Yeo 
Sent: September 13, 2018 8:15 PM
To: solr-user@lucene.apache.org
Subject: Re: Faceting with EnumFieldType in 7.1

Was the document re-indexed in Solr 7.1?

Regards,
Edwin

On Wed, 12 Sep 2018 at 23:38, Peter Tyrrell  wrote:

> I updated an older Solr 4.10 core to Solr 7.1 recently. In so doing, I 
> took an old 'gradeLevel_enum' field of type EnumField and made it an 
> EnumFieldType, since the former has been deprecated. The old core was 
> able to facet on gradeLevel_enum, but the new 7.1 core just returns no 
> facet values whatsoever for that field. Both cores return 
> gradeLevel_enum values ok when fl=gradeLevel_enum.
>
> In the schema, gradeLevel_enum is defined dynamically:
>
>  multiValued="true" />
>  enumsConfig="enumsConfig.xml" enumName="gradeLevels">
>
> This simple query fails to return any facet values in 7.1, but does 
> facet in 4.10:
>
>
> http://localhost:8983/solr/core1/select?facet.field=gradeLevel_enum
> cet=on=id,gradeLevel_enum=*:*=json
>
> Thanks for any insight.
>
> Peter Tyrrell, MLIS
> Lead Developer at Andornot
> 1-866-266-2525 x706 /
> ptyrr...@andornot.com<mailto:ptyrr...@andornot.com>
>
>

RE: Faceting with EnumFieldType in 7.1

[Peter Tyrrell]

Yes.

Peter Tyrrell, MLIS
Lead Developer at Andornot
1-866-266-2525 x706 / ptyrr...@andornot.com

-Original Message-
From: Zheng Lin Edwin Yeo  
Sent: September 13, 2018 8:15 PM
To: solr-user@lucene.apache.org
Subject: Re: Faceting with EnumFieldType in 7.1

Was the document re-indexed in Solr 7.1?

Regards,
Edwin

On Wed, 12 Sep 2018 at 23:38, Peter Tyrrell  wrote:

> I updated an older Solr 4.10 core to Solr 7.1 recently. In so doing, I 
> took an old 'gradeLevel_enum' field of type EnumField and made it an 
> EnumFieldType, since the former has been deprecated. The old core was 
> able to facet on gradeLevel_enum, but the new 7.1 core just returns no 
> facet values whatsoever for that field. Both cores return 
> gradeLevel_enum values ok when fl=gradeLevel_enum.
>
> In the schema, gradeLevel_enum is defined dynamically:
>
>  multiValued="true" />
>  enumsConfig="enumsConfig.xml" enumName="gradeLevels">
>
> This simple query fails to return any facet values in 7.1, but does 
> facet in 4.10:
>
>
> http://localhost:8983/solr/core1/select?facet.field=gradeLevel_enum
> cet=on=id,gradeLevel_enum=*:*=json
>
> Thanks for any insight.
>
> Peter Tyrrell, MLIS
> Lead Developer at Andornot
> 1-866-266-2525 x706 / 
> ptyrr...@andornot.com<mailto:ptyrr...@andornot.com>
>
>

Re: Faceting with EnumFieldType in 7.1

[Zheng Lin Edwin Yeo]

Was the document re-indexed in Solr 7.1?

Regards,
Edwin

On Wed, 12 Sep 2018 at 23:38, Peter Tyrrell  wrote:

> I updated an older Solr 4.10 core to Solr 7.1 recently. In so doing, I
> took an old 'gradeLevel_enum' field of type EnumField and made it an
> EnumFieldType, since the former has been deprecated. The old core was able
> to facet on gradeLevel_enum, but the new 7.1 core just returns no facet
> values whatsoever for that field. Both cores return gradeLevel_enum values
> ok when fl=gradeLevel_enum.
>
> In the schema, gradeLevel_enum is defined dynamically:
>
>  multiValued="true" />
>  enumsConfig="enumsConfig.xml" enumName="gradeLevels">
>
> This simple query fails to return any facet values in 7.1, but does facet
> in 4.10:
>
>
> http://localhost:8983/solr/core1/select?facet.field=gradeLevel_enum=on=id,gradeLevel_enum=*:*=json
>
> Thanks for any insight.
>
> Peter Tyrrell, MLIS
> Lead Developer at Andornot
> 1-866-266-2525 x706 / ptyrr...@andornot.com
>
>

Faceting with EnumFieldType in 7.1

[Peter Tyrrell]

I updated an older Solr 4.10 core to Solr 7.1 recently. In so doing, I took an 
old 'gradeLevel_enum' field of type EnumField and made it an EnumFieldType, 
since the former has been deprecated. The old core was able to facet on 
gradeLevel_enum, but the new 7.1 core just returns no facet values whatsoever 
for that field. Both cores return gradeLevel_enum values ok when 
fl=gradeLevel_enum.

In the schema, gradeLevel_enum is defined dynamically:




This simple query fails to return any facet values in 7.1, but does facet in 
4.10:

http://localhost:8983/solr/core1/select?facet.field=gradeLevel_enum=on=id,gradeLevel_enum=*:*=json

Thanks for any insight.

Peter Tyrrell, MLIS
Lead Developer at Andornot
1-866-266-2525 x706 / ptyrr...@andornot.com

Re: Solr range faceting

[Erick Erickson]

Oh, indeed if you changed anything about the field definition and did
_not_ reindex everything from scratch your results are unreliable.

I'm overstating the case, but it'll do for now. For instance, your
facet counts should be find if you changed the stored parameter for
instance, but anything else

And it's best to index into a new collection (or core in stand-alone)
rather then re-use the current one just for paranoia's sake.

Best,
Erick
On Fri, Sep 7, 2018 at 12:18 AM Dwane Hall  wrote:
>
> Thanks Erick,
>
> The field is defined as a pfloat.
>
>  indexed="true" stored="true"/>
>
> I took your advice and tried smaller result range and the counts look good.  
> I might try an index rebuild I’m wondering if the data has somehow been 
> corrupted by a combination of old and new index mappings.
>
> Thanks again for your assistance.
>
>
> "responseHeader":{
> "zkConnected":true,
> "status":0,
> "QTime":7},
>   "response":{"numFound":3,"start":0,"docs":[
>   {
> "Value":34.0},
>   {
> "Value":34.0},
>   {
> "Value":34.0}]
>   },
>   "facet_counts":
> "facet_queries":{},
> "facet_ranges":{
>   "Value:{
> "counts":[
>   "0.0",3],
>     "gap":100.0,
> "before":0,
> "after":0,
> "between":3,
> "start":0.0,
> "end":2000.0}}
> 
> From: Erick Erickson 
> Sent: Friday, 7 September 2018 12:54:48 PM
> To: solr-user
> Subject: Re: Solr range faceting
>
> Indeed this doesn't look right. By my count, you're missing 599 counts
> you'd expect in that range, although the after and between numbers
> total the numFound.
>
> What kind of a field is Value? Given the number of docs missing, I'd
> guess you could get the number of docs down really small and post
> them. Something like
> values 1, 2, 3, 4, 5, 
> and your range query so we could try it.
>
> What is the fieldType definition and field for Value?
>
> And finally, do you get different results if you use json faceting?
>
> Best,
> Erick
> On Thu, Sep 6, 2018 at 5:51 PM Dwane Hall  wrote:
> >
> > Thanks Jan that has fixed the bucket issue but I'm a little confused at why 
> > zero counts exist for some buckets when they appear to be values in them?
> >
> > "response":{"numFound":869,"start":0,"docs":[
> >   {
> > "Value":9475.08},
> >   {
> > "Value":780.0},
> >   {
> > "Value":9475.08},
> >   {
> > "Value":1000.0},
> >   {
> > "Value":50.0},
> >   {
> > "Value":50.0},
> >   {
> > "Value":0.0},
> >   {
> > "Value":800.0},
> >   {
> > "Value":0.0},
> >   {
> > "Value":1000.0},
> >   {
> > "Value":1000.0},
> >   {
> > "Value":5000.0},
> >   {
> > "Value":2000.0},
> >   {
> >"Value":4000.0},
> >   {
> > "Value":1500.0},
> >   {
> > "Value":0.0},
> >   {
> > "Value":1.0},
> >   {
> > "Value":5000.0},
> >   {
> > "Value":1000.0},
> >   {
> > "Value":0.0},
> >   {
> > "Value":1200.0},
> >   {
> > "Value":9000.0},
> >   {
> > "Value":1500.0},
> >   {
> > "Value":1.0},
> >   {
> > "Value":5000.0},
> >   {
> > "Value":4000.0},
> >   {
> > "Value":5000.0},
> >   {
> > "Value":5000.0},
> >   {
> > "Value":1.0},
> >   {
> > "Value":1000.0}]
> >   },
> >
> >   "facet_counts":{
> > "facet_queries":{},
> > "facet_ranges":{
> >   "Value":{
> > "counts":[
> >   "0.0",9,
> > 

Re: Solr range faceting

[Dwane Hall]

Thanks Erick,

The field is defined as a pfloat.



I took your advice and tried smaller result range and the counts look good.  I 
might try an index rebuild I’m wondering if the data has somehow been corrupted 
by a combination of old and new index mappings.

Thanks again for your assistance.


"responseHeader":{
"zkConnected":true,
"status":0,
"QTime":7},
  "response":{"numFound":3,"start":0,"docs":[
  {
"Value":34.0},
  {
"Value":34.0},
  {
"Value":34.0}]
  },
  "facet_counts":
"facet_queries":{},
"facet_ranges":{
  "Value:{
"counts":[
  "0.0",3],
"gap":100.0,
"before":0,
"after":0,
    "between":3,
"start":0.0,
"end":2000.0}}

From: Erick Erickson 
Sent: Friday, 7 September 2018 12:54:48 PM
To: solr-user
Subject: Re: Solr range faceting

Indeed this doesn't look right. By my count, you're missing 599 counts
you'd expect in that range, although the after and between numbers
total the numFound.

What kind of a field is Value? Given the number of docs missing, I'd
guess you could get the number of docs down really small and post
them. Something like
values 1, 2, 3, 4, 5, 
and your range query so we could try it.

What is the fieldType definition and field for Value?

And finally, do you get different results if you use json faceting?

Best,
Erick
On Thu, Sep 6, 2018 at 5:51 PM Dwane Hall  wrote:
>
> Thanks Jan that has fixed the bucket issue but I'm a little confused at why 
> zero counts exist for some buckets when they appear to be values in them?
>
> "response":{"numFound":869,"start":0,"docs":[
>   {
> "Value":9475.08},
>   {
> "Value":780.0},
>   {
> "Value":9475.08},
>   {
> "Value":1000.0},
>   {
> "Value":50.0},
>   {
> "Value":50.0},
>   {
> "Value":0.0},
>   {
> "Value":800.0},
>   {
> "Value":0.0},
>   {
> "Value":1000.0},
>   {
> "Value":1000.0},
>   {
> "Value":5000.0},
>   {
> "Value":2000.0},
>   {
>"Value":4000.0},
>   {
> "Value":1500.0},
>   {
> "Value":0.0},
>   {
> "Value":1.0},
>   {
> "Value":5000.0},
>   {
> "Value":1000.0},
>   {
> "Value":0.0},
>   {
> "Value":1200.0},
>   {
> "Value":9000.0},
>   {
> "Value":1500.0},
>   {
> "Value":1.0},
>   {
> "Value":5000.0},
>   {
> "Value":4000.0},
>   {
> "Value":5000.0},
>   {
> "Value":5000.0},
>   {
> "Value":1.0},
>   {
> "Value":1000.0}]
>   },
>
>   "facet_counts":{
> "facet_queries":{},
> "facet_ranges":{
>   "Value":{
> "counts":[
>   "0.0",9,
>   "100.0",0,
>   "200.0",0,
>   "300.0",0,
>   "400.0",80,
>   "500.0",0,
>   "600.0",0,
>   "700.0",69,
>   "800.0",0,
>   "900.0",0,
>   "1000.0",0,
>   "1100.0",0,
>   "1200.0",0,
>   "1300.0",0,
>   "1400.0",0,
>   "1500.0",0,
>   "1600.0",0,
>   "1700.0",0,
>   "1800.0",0,
>   "1900.0",9],
> "gap":100.0,
> "before":0,
> "after":103,
> "between":766,
> "start":0.0,
> "end":2000.0}
>
> Cheers,
>
> Dwane
> 
> From: Jan H?ydahl 
> Sent: Friday, 7 September 2018 9:23:44 AM
> To: solr-user@lucene.apache.org
> Subject: Re: Solr range faceting
>
> Try facet.minCount=0
>
> Jan
>
> > 7. sep. 2018 kl. 01:07 skrev Dwane 

Re: Solr range faceting

[Erick Erickson]

Indeed this doesn't look right. By my count, you're missing 599 counts
you'd expect in that range, although the after and between numbers
total the numFound.

What kind of a field is Value? Given the number of docs missing, I'd
guess you could get the number of docs down really small and post
them. Something like
values 1, 2, 3, 4, 5, 
and your range query so we could try it.

What is the fieldType definition and field for Value?

And finally, do you get different results if you use json faceting?

Best,
Erick
On Thu, Sep 6, 2018 at 5:51 PM Dwane Hall  wrote:
>
> Thanks Jan that has fixed the bucket issue but I'm a little confused at why 
> zero counts exist for some buckets when they appear to be values in them?
>
> "response":{"numFound":869,"start":0,"docs":[
>   {
> "Value":9475.08},
>   {
> "Value":780.0},
>   {
> "Value":9475.08},
>   {
> "Value":1000.0},
>   {
> "Value":50.0},
>   {
> "Value":50.0},
>   {
> "Value":0.0},
>   {
> "Value":800.0},
>   {
> "Value":0.0},
>   {
> "Value":1000.0},
>   {
> "Value":1000.0},
>   {
> "Value":5000.0},
>   {
> "Value":2000.0},
>   {
>"Value":4000.0},
>   {
> "Value":1500.0},
>   {
> "Value":0.0},
>   {
> "Value":1.0},
>   {
> "Value":5000.0},
>   {
> "Value":1000.0},
>   {
> "Value":0.0},
>   {
> "Value":1200.0},
>   {
> "Value":9000.0},
>   {
> "Value":1500.0},
>   {
> "Value":1.0},
>   {
> "Value":5000.0},
>   {
> "Value":4000.0},
>   {
> "Value":5000.0},
>   {
> "Value":5000.0},
>   {
> "Value":1.0},
>   {
> "Value":1000.0}]
>   },
>
>   "facet_counts":{
> "facet_queries":{},
> "facet_ranges":{
>   "Value":{
> "counts":[
>   "0.0",9,
>   "100.0",0,
>   "200.0",0,
>   "300.0",0,
>   "400.0",80,
>   "500.0",0,
>   "600.0",0,
>   "700.0",69,
>   "800.0",0,
>   "900.0",0,
>   "1000.0",0,
>   "1100.0",0,
>   "1200.0",0,
>   "1300.0",0,
>   "1400.0",0,
>   "1500.0",0,
>   "1600.0",0,
>   "1700.0",0,
>   "1800.0",0,
>   "1900.0",9],
> "gap":100.0,
> "before":0,
> "after":103,
> "between":766,
> "start":0.0,
> "end":2000.0}
>
> Cheers,
>
> Dwane
> 
> From: Jan H?ydahl 
> Sent: Friday, 7 September 2018 9:23:44 AM
> To: solr-user@lucene.apache.org
> Subject: Re: Solr range faceting
>
> Try facet.minCount=0
>
> Jan
>
> > 7. sep. 2018 kl. 01:07 skrev Dwane Hall :
> >
> > Good morning Solr community.  I'm having a few facet range issues for which 
> > I'd appreciate some advice when somebody gets a spare couple of minutes.
> >
> > Environment
> > Solr Cloud (7.3.1)
> > Single Shard Index, No replicas
> >
> > Facet Configuration (I'm using the request params API and useParams at 
> > runtime)
> > "facet":"true",
> > "facet.mincount":1,
> > "facet.missing":"false",
> > "facet.range":"Value"
> > "f.Value.facet.range.start":0.0,
> > "f.Value.facet.range.end":2000.0,
> > "f.Value.facet.range.gap":100,
> > "f.Value.facet.range.include":"edge",
> > "f.Value.facet.range.other":"all",
> >
> > My problem
> > With my range facet configuration I'm expecting to see a facet range entry 
> > for every 'step' (100 in my case) between my facet.range.start and 
> > facet.range.end s

Re: Solr range faceting

[Dwane Hall]

Thanks Jan that has fixed the bucket issue but I'm a little confused at why 
zero counts exist for some buckets when they appear to be values in them?

"response":{"numFound":869,"start":0,"docs":[
  {
"Value":9475.08},
  {
"Value":780.0},
  {
"Value":9475.08},
  {
"Value":1000.0},
  {
"Value":50.0},
  {
"Value":50.0},
  {
"Value":0.0},
  {
"Value":800.0},
  {
"Value":0.0},
  {
"Value":1000.0},
  {
"Value":1000.0},
  {
"Value":5000.0},
  {
"Value":2000.0},
  {
   "Value":4000.0},
  {
"Value":1500.0},
  {
"Value":0.0},
  {
"Value":1.0},
  {
"Value":5000.0},
  {
"Value":1000.0},
  {
"Value":0.0},
  {
"Value":1200.0},
  {
"Value":9000.0},
  {
"Value":1500.0},
  {
"Value":1.0},
  {
"Value":5000.0},
  {
"Value":4000.0},
  {
"Value":5000.0},
  {
"Value":5000.0},
  {
"Value":1.0},
  {
"Value":1000.0}]
  },

  "facet_counts":{
"facet_queries":{},
"facet_ranges":{
  "Value":{
"counts":[
  "0.0",9,
  "100.0",0,
  "200.0",0,
  "300.0",0,
  "400.0",80,
  "500.0",0,
  "600.0",0,
  "700.0",69,
  "800.0",0,
  "900.0",0,
  "1000.0",0,
  "1100.0",0,
  "1200.0",0,
  "1300.0",0,
  "1400.0",0,
  "1500.0",0,
  "1600.0",0,
  "1700.0",0,
  "1800.0",0,
  "1900.0",9],
"gap":100.0,
"before":0,
"after":103,
"between":766,
"start":0.0,
"end":2000.0}

Cheers,

Dwane

From: Jan H?ydahl 
Sent: Friday, 7 September 2018 9:23:44 AM
To: solr-user@lucene.apache.org
Subject: Re: Solr range faceting

Try facet.minCount=0

Jan

> 7. sep. 2018 kl. 01:07 skrev Dwane Hall :
>
> Good morning Solr community.  I'm having a few facet range issues for which 
> I'd appreciate some advice when somebody gets a spare couple of minutes.
>
> Environment
> Solr Cloud (7.3.1)
> Single Shard Index, No replicas
>
> Facet Configuration (I'm using the request params API and useParams at 
> runtime)
> "facet":"true",
> "facet.mincount":1,
> "facet.missing":"false",
> "facet.range":"Value"
> "f.Value.facet.range.start":0.0,
> "f.Value.facet.range.end":2000.0,
> "f.Value.facet.range.gap":100,
> "f.Value.facet.range.include":"edge",
> "f.Value.facet.range.other":"all",
>
> My problem
> With my range facet configuration I'm expecting to see a facet range entry 
> for every 'step' (100 in my case) between my facet.range.start and 
> facet.range.end settings. Something like the following 0.0,100.0,200.0, 
> ...2000.0 with a sum of the number of values that occur between each range 
> step.  This does not appear to be the case and in some instances I don't 
> appear to get counts for some range steps (800.0 and 1000.0 for example are 
> present in my result set range below but I don't get a range value facets for 
> these values?)
>
> Am I completely misunderstanding how range facets are supposed to work or is 
> my configuration a little askew?
>
> Any advice would be greatly appreciated.
>
> The Solr Response
> "responseHeader":{
>"zkConnected":true,
>"status":0,
>"QTime":121},
>
>  "response":{"numFound":869,"start":0,"docs":[
>  {
>"Value":9475.08},
>  {
>"Value":780.0},
>  {
>"Value":1000.0},
>  {
>"Value":50.0},
>  {
>"Value":50.0},
>  {
>"Value":0.0},
>  {
>"Value":800.0},
>  {
>"Value":0.0},
>  {
>"Value":1000.0},
>  {
>"Value":1000.0},
>  {
>"Value":5000.0},
>  {
>"Value":2000.0},
>  {
>"Value":4000.0},
>  {
>"Value":1500.0},
>  {
>"Value":0.0},
>  {
>"Value":1.0},
>  {
>"Value":1000.0}]
>  },
>  "facet_counts":{
>"facet_ranges":{
>  "Value":{
>"counts":[
>  "0.0",9,
>  "400.0",80,
>  "700.0",69,
>  "1900.0",9],
>"gap":100.0,
>"before":0,
>"after":103,
>"between":766,
>"start":0.0,
>"end":2000.0}}
>
> Cheers,
>
> Dwane

Re: Solr range faceting

[Jan Høydahl]

Try facet.minCount=0

Jan

> 7. sep. 2018 kl. 01:07 skrev Dwane Hall :
> 
> Good morning Solr community.  I'm having a few facet range issues for which 
> I'd appreciate some advice when somebody gets a spare couple of minutes.
> 
> Environment
> Solr Cloud (7.3.1)
> Single Shard Index, No replicas
> 
> Facet Configuration (I'm using the request params API and useParams at 
> runtime)
> "facet":"true",
> "facet.mincount":1,
> "facet.missing":"false",
> "facet.range":"Value"
> "f.Value.facet.range.start":0.0,
> "f.Value.facet.range.end":2000.0,
> "f.Value.facet.range.gap":100,
> "f.Value.facet.range.include":"edge",
> "f.Value.facet.range.other":"all",
> 
> My problem
> With my range facet configuration I'm expecting to see a facet range entry 
> for every 'step' (100 in my case) between my facet.range.start and 
> facet.range.end settings. Something like the following 0.0,100.0,200.0, 
> ...2000.0 with a sum of the number of values that occur between each range 
> step.  This does not appear to be the case and in some instances I don't 
> appear to get counts for some range steps (800.0 and 1000.0 for example are 
> present in my result set range below but I don't get a range value facets for 
> these values?)
> 
> Am I completely misunderstanding how range facets are supposed to work or is 
> my configuration a little askew?
> 
> Any advice would be greatly appreciated.
> 
> The Solr Response
> "responseHeader":{
>"zkConnected":true,
>"status":0,
>"QTime":121},
> 
>  "response":{"numFound":869,"start":0,"docs":[
>  {
>"Value":9475.08},
>  {
>"Value":780.0},
>  {
>"Value":1000.0},
>  {
>"Value":50.0},
>  {
>"Value":50.0},
>  {
>"Value":0.0},
>  {
>"Value":800.0},
>  {
>"Value":0.0},
>  {
>"Value":1000.0},
>  {
>"Value":1000.0},
>  {
>"Value":5000.0},
>  {
>"Value":2000.0},
>  {
>"Value":4000.0},
>  {
>"Value":1500.0},
>  {
>"Value":0.0},
>  {
>"Value":1.0},
>  {
>"Value":1000.0}]
>  },
>  "facet_counts":{
>"facet_ranges":{
>  "Value":{
>"counts":[
>  "0.0",9,
>  "400.0",80,
>  "700.0",69,
>  "1900.0",9],
>"gap":100.0,
>"before":0,
>"after":103,
>"between":766,
>"start":0.0,
>"end":2000.0}}
> 
> Cheers,
> 
> Dwane

Solr range faceting

[Dwane Hall]

Good morning Solr community.  I'm having a few facet range issues for which I'd 
appreciate some advice when somebody gets a spare couple of minutes.

Environment
Solr Cloud (7.3.1)
Single Shard Index, No replicas

Facet Configuration (I'm using the request params API and useParams at runtime)
"facet":"true",
"facet.mincount":1,
"facet.missing":"false",
"facet.range":"Value"
"f.Value.facet.range.start":0.0,
"f.Value.facet.range.end":2000.0,
"f.Value.facet.range.gap":100,
"f.Value.facet.range.include":"edge",
"f.Value.facet.range.other":"all",

My problem
With my range facet configuration I'm expecting to see a facet range entry for 
every 'step' (100 in my case) between my facet.range.start and facet.range.end 
settings. Something like the following 0.0,100.0,200.0, ...2000.0 with a sum of 
the number of values that occur between each range step.  This does not appear 
to be the case and in some instances I don't appear to get counts for some 
range steps (800.0 and 1000.0 for example are present in my result set range 
below but I don't get a range value facets for these values?)

Am I completely misunderstanding how range facets are supposed to work or is my 
configuration a little askew?

Any advice would be greatly appreciated.

The Solr Response
"responseHeader":{
"zkConnected":true,
"status":0,
"QTime":121},

  "response":{"numFound":869,"start":0,"docs":[
  {
"Value":9475.08},
  {
"Value":780.0},
  {
"Value":1000.0},
  {
"Value":50.0},
  {
"Value":50.0},
  {
"Value":0.0},
  {
"Value":800.0},
  {
"Value":0.0},
  {
"Value":1000.0},
  {
"Value":1000.0},
  {
"Value":5000.0},
  {
"Value":2000.0},
  {
"Value":4000.0},
  {
"Value":1500.0},
  {
"Value":0.0},
  {
"Value":1.0},
  {
"Value":1000.0}]
  },
  "facet_counts":{
"facet_ranges":{
  "Value":{
"counts":[
  "0.0",9,
  "400.0",80,
  "700.0",69,
  "1900.0",9],
"gap":100.0,
"before":0,
"after":103,
"between":766,
"start":0.0,
"end":2000.0}}

Cheers,

Dwane

Re: Faceting with nested Document

[Mikhail Khludnev]

The first two mistakes are:
 - using fq for children fields ,
 - using a value master_id:0 as a parents' filter
Regarding the question, you are getting non-zero facets because you exclude
filter produces empty results.

Faceting with nested Document

[Rajesh Kumar]

Hi All,

I am trying to do faceting with "tag and exclude" on nested document
Below nested document sample
{"master_id":"8219",
"color_label":["White"],
"price":1550.0,
"product_id":"8220",
"size_label":["XS"],
"color_option_id":["82"],
"product":"PomPom Summer Dress",
"child_sku":"VJ17MSCL02MAY-XS",
"size_option_id":["94"],}

  {
"title":"PomPom Summer Dress",
"color_label":["White"],
"price":1550.0,
"product_id":"8219",
"color_option_id":["82"],
"product":"PomPom Summer Dress",
"master_sku":"VJ17MSCL02MAY",
"master_id":"0",}
My objective is to get count of products for the field like
color_label,size_label then filter query on these fields.Below is the query
which i m using to get products with size label "L" with faceting
query?q=title:pant={!tag=label}size_label:L=size_label,color_label,id,[child
parentFilter=master_id:0]&
json.facet={
size_label :
{type: terms,
field: size_label,
domain: { blockChildren:"master_id:0",excludeTags:label}
}
}

"response":{"numFound":0,"start":0,"docs":[]
  },
  "facets":{
"count":0,
"size_label":{
  "buckets":[{
  "val":"L",
  "count":39},
{
  "val":"M",
  "count":39},
{
  "val":"XS",
  "count":30},
{
  "val":"XXL",
  "count":30}]}}}

here is no product in response (numFound=0) but still getting "facets"
node. Please help me to understand where i am doing wrong

Regards
Rajesh Kumar

Re: Block Join Faceting in Solr 7.2

[Mikhail Khludnev]

 uniqueBlock is not faster than BlockJoinFacet in 7.4.

On Tue, Aug 7, 2018 at 8:05 AM Aditya Gandhi  wrote:

> I'm querying an Index which has two types of child documents (let's call
> them ChildTypeA and ChildTypeB)
> I wrap the subqueries for each of these documents in a boolean clause,
> something like this:
>
> *q=+{! parent which=type:parent } +{! parent
> which=type:parent }*
>
>
> I've been trying to get facet counts on documents of ChildTypeA (rolled up
> by parent) and I've tried the following approaches
>
>
>- Tried Block Join Faceting using the JSON API  i.e. using the
>unique(_root_) approach.
>   -  Enabled docValues on _root_
>   - *This did not scale well*
>- Tried using the BlockJoinFacet component.
>   - Had to customize it since it expects that only one
>   *ToParentBlockJoinQuery* clause to be present in the query.
>   - Since I needed facet counts only on ChildTypeA, I changed it to
>   ignore the clause on ChildTypeB
>   - I did not enable docValues on _root_ since it was not mentioned in
>   the documentation.
>   - *This approach did not scale well*
>
> I needed advice on whether I could have done anything better in any one of
> the two approached I've tried so far. Also if there exists some other
> approached I could try.
> Would using the uniqueBlock in 7.4 help? (Though this would require me to
> upgrade my Solr version)
>


-- 
Sincerely yours
Mikhail Khludnev

Block Join Faceting in Solr 7.2

[Aditya Gandhi]

I'm querying an Index which has two types of child documents (let's call
them ChildTypeA and ChildTypeB)
I wrap the subqueries for each of these documents in a boolean clause,
something like this:

*q=+{! parent which=type:parent } +{! parent
which=type:parent }*


I've been trying to get facet counts on documents of ChildTypeA (rolled up
by parent) and I've tried the following approaches


   - Tried Block Join Faceting using the JSON API  i.e. using the
   unique(_root_) approach.
  -  Enabled docValues on _root_
  - *This did not scale well*
   - Tried using the BlockJoinFacet component.
  - Had to customize it since it expects that only one
  *ToParentBlockJoinQuery* clause to be present in the query.
  - Since I needed facet counts only on ChildTypeA, I changed it to
  ignore the clause on ChildTypeB
  - I did not enable docValues on _root_ since it was not mentioned in
  the documentation.
  - *This approach did not scale well*

I needed advice on whether I could have done anything better in any one of
the two approached I've tried so far. Also if there exists some other
approached I could try.
Would using the uniqueBlock in 7.4 help? (Though this would require me to
upgrade my Solr version)

Single Filter tagged with multiple tags in Multi Select Faceting

[Zheng Lin Edwin Yeo]

Hi,

Regarding the Multi Select Faceting, there is a feature whereby we can have
a single filter tagged with multiple tags, like the following example:
fq={!tag=tag1,tag2,tag3}my_field:my_filter

However, I have not been able to figure out the use and advantage of this.
Anyone has any information on this? I am using Solr 7.4.0

Regards,
Edwin

Re: Block Join Faceting issue

[Mikhail Khludnev]

 q=*:*={!parent which="doc_type:parent"}+doc_type:child -child_color:*

Make sure that + isn;t grabbed by url encoding.

On Wed, Jul 25, 2018 at 8:23 AM sagandhi  wrote:

> Hi Mikhail,
>
> Thank you for suggesting to use json facet. I tried json.facet, it works
> great and I am able to make a single query instead of two. Now I am
> planning
> to get rid of the duplicate child fields in parent docs. However I ran into
> problems while forming negative queries with block join.
>
> Here's what I would like to query - Get me parent docs whose children do
> not
> have a particular field.
> I tried these but none worked -
>
> q=*:*={!parent which="doc_type:parent"}*-*child_color:*
> q=*:*={!parent which="doc_type:parent" v=$qq}=(!child_color:*)
>
> Currently I have duplicate entries of child fields in parent docs, so I am
> able to do this -
> =!parent_color:*
>
> Is there a way to form this query using block join?
>
> Thanks,
> Soham
>
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>


-- 
Sincerely yours
Mikhail Khludnev

Re: Block Join Faceting issue

[sagandhi]

Hi Mikhail,

Thank you for suggesting to use json facet. I tried json.facet, it works
great and I am able to make a single query instead of two. Now I am planning
to get rid of the duplicate child fields in parent docs. However I ran into
problems while forming negative queries with block join.

Here's what I would like to query - Get me parent docs whose children do not
have a particular field.
I tried these but none worked - 

q=*:*={!parent which="doc_type:parent"}*-*child_color:*
q=*:*={!parent which="doc_type:parent" v=$qq}=(!child_color:*)

Currently I have duplicate entries of child fields in parent docs, so I am
able to do this - 
=!parent_color:*

Is there a way to form this query using block join? 

Thanks,
Soham




--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Block Join Faceting issue

[soham gandhi]

I am using solr 6.2.1. Is json.facet supported there? If not, is there a
way to combine the 2 queries into a single query in solr 6.2.1? In the
example I mentioned, could you please help me form a sample query where I
can facet on both parent as well as child docs?

Thanks,
Soham

On Wed, Jul 18, 2018 at 1:35 PM, Mikhail Khludnev  wrote:

>  child.facet.field  works if parent query goes as q ie q={!parent
> which="doc_type:parent"}color:blue.
> However, it's considered as deprecated and proposed to be replaced with
> json.facet and uniqueBlock
>
> On Wed, Jul 18, 2018 at 7:04 AM soham gandhi 
> wrote:
>
> > Hi,
> >
> > I am working on a query that must return parent docs and facet on both
> the
> > parent and child fields. Here's a sample doc-
> > 
> > 1
> > merc
> > car
> > sagandhi
> > 2
> > 3
> > parent
> > 
> > 2
> > child
> > S
> > blue
> > 
> > 
> > 3
> > Z
> > 25
> > child
> > 
> > 
> > I want to search for "merc" or "*" and return facets for type/user from
> > parent docs, and code/color from child docs. Currently I am not using
> > blockjoin. Instead I make two queries, one on the parent docs and the
> other
> > on the child docs. I get the applicable child_id from the first query and
> > feed it into the second query to get the child facets.
> > However this has impacted performance and is not scalable if the
> child_ids
> > I get are huge. Is there a way to combine the two queries using block
> join.
> > I tried this query -
> > q=*:* =user:sagandhi ={!parent
> > which="doc_type:parent"}color:blue=type
> facet.field=code
> >
> > I get this error - Block join faceting is allowed with
> > ToParentBlockJoinQuery only
> >
> > Am I missing something here? Any pointers please.
> >
> > Thanks,
> > Soham
> >
>
>
> --
> Sincerely yours
> Mikhail Khludnev
>

Re: Block Join Faceting issue

[Mikhail Khludnev]

 child.facet.field  works if parent query goes as q ie q={!parent
which="doc_type:parent"}color:blue.
However, it's considered as deprecated and proposed to be replaced with
json.facet and uniqueBlock

On Wed, Jul 18, 2018 at 7:04 AM soham gandhi 
wrote:

> Hi,
>
> I am working on a query that must return parent docs and facet on both the
> parent and child fields. Here's a sample doc-
> 
> 1
> merc
> car
> sagandhi
> 2
> 3
> parent
> 
> 2
> child
> S
> blue
> 
> 
> 3
> Z
> 25
> child
> 
> 
> I want to search for "merc" or "*" and return facets for type/user from
> parent docs, and code/color from child docs. Currently I am not using
> blockjoin. Instead I make two queries, one on the parent docs and the other
> on the child docs. I get the applicable child_id from the first query and
> feed it into the second query to get the child facets.
> However this has impacted performance and is not scalable if the child_ids
> I get are huge. Is there a way to combine the two queries using block join.
> I tried this query -
> q=*:* =user:sagandhi ={!parent
> which="doc_type:parent"}color:blue=type=code
>
> I get this error - Block join faceting is allowed with
> ToParentBlockJoinQuery only
>
> Am I missing something here? Any pointers please.
>
> Thanks,
> Soham
>


-- 
Sincerely yours
Mikhail Khludnev

Block Join Faceting issue

[soham gandhi]

Hi,

I am working on a query that must return parent docs and facet on both the
parent and child fields. Here's a sample doc-

1
merc
car
sagandhi
2
3
parent

2
child
S
blue


3
Z
25
child


I want to search for "merc" or "*" and return facets for type/user from
parent docs, and code/color from child docs. Currently I am not using
blockjoin. Instead I make two queries, one on the parent docs and the other
on the child docs. I get the applicable child_id from the first query and
feed it into the second query to get the child facets.
However this has impacted performance and is not scalable if the child_ids
I get are huge. Is there a way to combine the two queries using block join.
I tried this query -
q=*:* =user:sagandhi ={!parent
which="doc_type:parent"}color:blue=type=code

I get this error - Block join faceting is allowed with
ToParentBlockJoinQuery only

Am I missing something here? Any pointers please.

Thanks,
Soham

Re: Faceting over ExternalFileField

[Mikhail Khludnev]

Absence of error is a bug for me. The problem is that eff is doubles not
strings with ordinals. It would be possible after
https://issues.apache.org/jira/browse/SOLR-10528
Now you can try to create several type:query subfacets passing either
{!frange} or just plain Lucene query (there is a slight chance they works
with eff).

On Wed, May 9, 2018 at 1:36 PM, Michal Danilák 
wrote:

> Is it possible to facet over ExternalFileField values?
>
> If I have this in my schema.xml:
>
>  keyField="id" defVal="0" stored="false" indexed="false" />
>
> 
>
> And request the following facet:
>
> facet={
> "age": {
> "field": "eff_age",
> "type": "terms",
> "limit": 10
> }
> }
>
> It returns an empty list of buckets.
>
> First, it doesn't throw an error, which means, it should do something.
> Second, if I'm not mistaken, external file fields behave like doc values,
> so it should be possible to facet over them.
>
> Am I doing something wrong? Is there some other way around this?
>
> Thanks
>
> -Michal
>



-- 
Sincerely yours
Mikhail Khludnev

Faceting over ExternalFileField

[Michal Danilák]

Is it possible to facet over ExternalFileField values?

If I have this in my schema.xml:





And request the following facet:

facet={
"age": {
"field": "eff_age",
"type": "terms",
"limit": 10
}
}

It returns an empty list of buckets.

First, it doesn't throw an error, which means, it should do something.
Second, if I'm not mistaken, external file fields behave like doc values,
so it should be possible to facet over them.

Am I doing something wrong? Is there some other way around this?

Thanks

-Michal

Re: Faceting question

[Shawn Heisey]

On 5/2/2018 2:56 PM, Weffelmeyer, Stacie wrote:
> Question on faceting.  We have a dynamicField that we want to facet
> on. Below is the field and the type of information that field generates.
>
>  
>
> cid:image001.png@01D3E22D.DE028870
>

This image is not available.  This mailing list will almost always strip
attachments from email that it receives.

>    
> "*customMetadata*":["{\"controlledContent\":{\"metadata\":{\"programs\":[\"program1\"],\"departments\":[\"department1\"],\"locations\":[\"location1\"],\"functions\":[\"function1\"],\"customTags\":[\"customTag1\",\"customTag2\"],\"corporate\":false,\"redline\":false},\"who\":{\"lastUpdateDate\":\"2018-04-26T14:35:02.268Z\",\"creationDate\":\"2018-04-26T14:35:01.445Z\",\"createdBy\":38853},\"clientOwners\":[38853],\"clientLastUpdateDate\":\"2018-04-25T21:15:06.000Z\",\"clientCreationDate\":\"2018-04-25T20:58:34.000Z\",\"clientContentId\":\"DOC-8030\",\"type\":{\"applicationId\":2574,\"code\":\"WI\",\"name\":\"Work
> Instruction\",\"id\":\"5ac3d4d111570f0047a8ceb9\"},\"status\":\"active\",\"version\":1}}"],
>

I do not know what this is.  It looks a little like JSON.  But if it's
json, there are a lot of escaped quotes in it, and I don't really know
what I'm looking at.

>  
>
> It will always have customMetadata.controlledContent.metadata
>
>  
>
> Then from metadata, it could be anything, which is why it is a
> dynamicField.
>
>  
>
> In this example there is
>
> customMetadata.controlledContent.metadata.programs
>
> customMetadata.controlledContent.metadata.departments
>
> customMetadata.controlledContent.metadata.locations
>

Solr does not have the concept of a nested data type.  So how are you
getting from all that text above to period-delimited strings in a
hierarchy?  If you're using some kind of custom plugin for Solr to have
it support something it doesn't do out of the box, you're probably going
to need to talk to the author of that plugin.

Solr's dynamicField support is only dynamic in the sense that the
precise field name is not found in the schema.  The field name is
dynamic.  When it comes to what's IN the field, it doesn't matter
whether it's a dynamic field or not.

> If I enable faceting, it will do so with the field customMetadata. But
> it doesn’t help because it separates every space as a term.  But
> ideally I want to facet on customMetadata.controlledContent.metadata.
> Doing so brings back no facets.
>
>  
>
> Is this possible?  How can we best accomplish this?
>

We will need to understand exactly what you are indexing, what's in your
schema, the exact query requests you are sending, and what you are
expecting back.

Thanks,
Shawn