[
https://issues.apache.org/jira/browse/SOLR-8362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Toke Eskildsen updated SOLR-8362:
---------------------------------
Comment: was deleted
(was: [~hossman] using this field type for distributed faceting can lead to
wrong results. Maybe this should be noted in the JavaDoc or the Solr
documentation?
This can be demonstrated by installing the cloud-version of the
{{gettingstarted}} sample with
{{./solr -e cloud}}
using defaults all the way, except for {{shards}} which should be {{3}}. After
that a corpus can be indexed with
{{( echo '[' ; for J in $(seq 0 99); do ID=$((J)) ; echo
"\{\"id\":\"$ID\",\"facet_t_sort\":\"a b $J\"},"; done ; echo
'\{"id":"duplicate_1","facet_t_sort":"a
b"},\{"id":"duplicate_2","facet_t_sort":"a b"}]' ) | curl -s -d @- -X POST -H
'Content-Type: application/json'
'http://localhost:8983/solr/gettingstarted/update?commit=true'}}
This will index 100 documents with a single-valued field {{facet_t_sort:"a b
X"}} where X is the document number + 2 documents with {{facet_t_sort:"a b"}}.
The call
curl
'http://localhost:8983/solr/gettingstarted/select?facet.field=facet_t_sort&facet.limit=5&facet=on&q=*:*&rows=0'
should return "a b" as the top facet term with count 2, but returns
{{{}}
{{ "responseHeader":{}}
{{ "zkConnected":true,}}
{{ "status":0,}}
{{ "QTime":13,}}
{{ "params":{}}
{{ "facet.limit":"5",}}
{{ "q":"*:*",}}
{{ "facet.field":"facet_t_sort",}}
{{ "rows":"0",}}
{{ "facet":"on"}},}}
{{ "response":{"numFound":102,"start":0,"maxScore":1.0,"docs":[]}}
{{ },}}
{{ "facet_counts":{}}
{{ "facet_queries":{},}}
{{ "facet_fields":{}}
{{ "facet_t_sort":[}}
{{ "a b",36,}}
{{ "a b 0",1,}}
{{ "a b 1",1,}}
{{ "a b 10",1,}}
{{ "a b 11",1]},}}
{{ "facet_ranges":{},}}
{{ "facet_intervals":{},}}
{{ "facet_heatmaps":{}}}}}
The problem is the second phase of simple faceting, where the fine-counting
happens. In the first phase, "a b" is returned from 1 or 2 of the 3 shards. It
wins the popularity contest as there are 2 "a b"-terms and only 1 of all the
other terms. The 1 or 2 shards that did not deliver "a b" in the first phase
are then queried for the count for "a b", which happens in the form of a
{{facet_t_sort:"a b"}}-lookup. It seems that this lookup uses the analyzer
chain and thus matches _all_ the documents in that shard (approximately 102/3).
An alternative would be to do the fine-counting on the DocValues instead, but
that works very poorly with many values, so that seems more like a trap than a
solution.)
> Add docValues support for TextField
> -----------------------------------
>
> Key: SOLR-8362
> URL: https://issues.apache.org/jira/browse/SOLR-8362
> Project: Solr
> Issue Type: Improvement
> Reporter: Hoss Man
> Priority: Major
>
> At the last lucene/solr revolution, Toke asked a question about why TextField
> doesn't support docValues. The short answer is because no one ever added it,
> but the longer answer was because we would have to think through carefully
> the _intent_ of supporting docValues for a "tokenized" field like TextField,
> and how to support various conflicting usecases where they could be handy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
