[jira] [Commented] (SOLR-12457) field(x,min|max) sorting doesn't work on trie or str fields in multi-shard collections
[
https://issues.apache.org/jira/browse/SOLR-12457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16510391#comment-16510391
]
Hoss Man commented on SOLR-12457:
-
Ok ... here's some spit balling...
ultimatley the root of the "ClassCastException: java.lang.String cannot be cast
to org.apache.lucene.util.BytesRef" comes down to the age old problem of
JavaBinCodec defaulting to {{writeVal(val.getClass().getName() + ':' +
val.toString());}} when it doesn't know how to write ab Object.
In particular: {{QueryComponent.doFieldSortValues}} tries to be agnostic to the
SortFields/Comparators used, and puts the "raw" sort value directly in the
response (unless there is a SchemaField that can be used to marshall it) .. in
the case of functions like these, that's a BytesRef -- on the other side of
things, the resulting "String" (ie:
{{org.apache.lucene.util.BytesRef:[DEADBEEF]}} ) gets read in and cases the
class cast when it's passed to TermOrdValComparator.
In an ideal world, the intra node communication would just happily "work" even
with {{BytesRef}} objects -- in which case we wouldn't even need the existing
marshall/unmarshall logic that StrField has -- and we could potentially
configure an ObjectResolver on the JavaBinCodec used for intra node
communication to do so ... but that seems like a pretty big change for this and
still wouldn't fix things for anyone still using xml for intra node
communication (do these people exist?)
a "hack" with a more limited surface area of change would be to make
{{QueryComponent.doFieldSortValues}} and {{QueryComponent.unmarshalSortValues}}
"smarter" such that they take care of automatically base64 encoding/decoding
sort values if/when the SortField uses a TermOrdValComparator
...that's really the best idea i've got.
> field(x,min|max) sorting doesn't work on trie or str fields in multi-shard
> collections
> --
>
> Key: SOLR-12457
> URL: https://issues.apache.org/jira/browse/SOLR-12457
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 7.1
>Reporter: Varun Thacker
>Priority: Major
> Attachments: SOLR-12457.patch
>
>
> When I go to sort on a multi-valued field in a 2 shard collection, which has
> trie fields the query fails.
> To reproduce we need 2+ shards, a multi-valued trie field and "desc" sort
> criteria.
> Here's my schema
> {code:java}
> multiValued="true" docValues="true"/>
> positionIncrementGap="0" precisionStep="0"/>
> multiValued="true"/>
>
> {code}
> Now If I add a few docs
> {code:java}
> [
> {"id" : "1", "test_is" : ["1", "2", "3", "4", "5"], "test_i" : ["1", "2",
> "3", "4", "5"]},
> {"id" : "2", "test_is" : ["1", "2", "3", "4", "5"], "test_i" : ["1", "2",
> "3", "4", "5"]},
> {"id" : "3", "test_is" : ["1", "2", "3", "4", "5"], "test_i" : ["1", "2",
> "3", "4", "5"]}
> ]{code}
> Works:
> [http://localhost:8983/solr/gettingstarted/select?q=*:*=field(test_i,max)%20desc]
>
> Doesn't Work:
> [http://localhost:8983/solr/gettingstarted/select?q=*:*=field(test_is,max)%20desc]
>
> To be more clear when I say it doesn't work , the query throws and error and
> here's the stack trace for it:
> {code:java}
> ERROR - 2018-06-06 22:55:06.599; [c:gettingstarted s:shard2 r:core_node8
> x:gettingstarted_shard2_replica_n5] org.apache.solr.common.SolrException;
> null:java.lang.ClassCastException: java.lang.String cannot be cast to
> org.apache.lucene.util.BytesRef
> at
> org.apache.lucene.search.FieldComparator$TermOrdValComparator.compareValues(FieldComparator.java:561)
> at
> org.apache.solr.handler.component.ShardFieldSortedHitQueue$1.compare(ShardFieldSortedHitQueue.java:161)
> at
> org.apache.solr.handler.component.ShardFieldSortedHitQueue$1.compare(ShardFieldSortedHitQueue.java:153)
> at
> org.apache.solr.handler.component.ShardFieldSortedHitQueue.lessThan(ShardFieldSortedHitQueue.java:91)
> at
> org.apache.solr.handler.component.ShardFieldSortedHitQueue.lessThan(ShardFieldSortedHitQueue.java:33)
> at org.apache.lucene.util.PriorityQueue.upHeap(PriorityQueue.java:263)
> at org.apache.lucene.util.PriorityQueue.add(PriorityQueue.java:140)
> at
> org.apache.lucene.util.PriorityQueue.insertWithOverflow(PriorityQueue.java:156)
> at
> org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:924)
> at
> org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:585)
> at
> org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:564)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:423)
> at
>
[jira] [Commented] (SOLR-12457) field(x,min|max) sorting doesn't work on trie or str fields in multi-shard collections
[
https://issues.apache.org/jira/browse/SOLR-12457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16510314#comment-16510314
]
Hoss Man commented on SOLR-12457:
-
attached a patch that converts {{SortByFunctionTest}} to a MiniSolrCloud based
test (NOTE: {{TestSortByMinMaxFunction extendsSortByFunctionTest}} ) as a
starting point for testing & fixing this
> field(x,min|max) sorting doesn't work on trie or str fields in multi-shard
> collections
> --
>
> Key: SOLR-12457
> URL: https://issues.apache.org/jira/browse/SOLR-12457
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 7.1
>Reporter: Varun Thacker
>Priority: Major
> Attachments: SOLR-12457.patch
>
>
> When I go to sort on a multi-valued field in a 2 shard collection, which has
> trie fields the query fails.
> To reproduce we need 2+ shards, a multi-valued trie field and "desc" sort
> criteria.
> Here's my schema
> {code:java}
> multiValued="true" docValues="true"/>
> positionIncrementGap="0" precisionStep="0"/>
> multiValued="true"/>
>
> {code}
> Now If I add a few docs
> {code:java}
> [
> {"id" : "1", "test_is" : ["1", "2", "3", "4", "5"], "test_i" : ["1", "2",
> "3", "4", "5"]},
> {"id" : "2", "test_is" : ["1", "2", "3", "4", "5"], "test_i" : ["1", "2",
> "3", "4", "5"]},
> {"id" : "3", "test_is" : ["1", "2", "3", "4", "5"], "test_i" : ["1", "2",
> "3", "4", "5"]}
> ]{code}
> Works:
> [http://localhost:8983/solr/gettingstarted/select?q=*:*=field(test_i,max)%20desc]
>
> Doesn't Work:
> [http://localhost:8983/solr/gettingstarted/select?q=*:*=field(test_is,max)%20desc]
>
> To be more clear when I say it doesn't work , the query throws and error and
> here's the stack trace for it:
> {code:java}
> ERROR - 2018-06-06 22:55:06.599; [c:gettingstarted s:shard2 r:core_node8
> x:gettingstarted_shard2_replica_n5] org.apache.solr.common.SolrException;
> null:java.lang.ClassCastException: java.lang.String cannot be cast to
> org.apache.lucene.util.BytesRef
> at
> org.apache.lucene.search.FieldComparator$TermOrdValComparator.compareValues(FieldComparator.java:561)
> at
> org.apache.solr.handler.component.ShardFieldSortedHitQueue$1.compare(ShardFieldSortedHitQueue.java:161)
> at
> org.apache.solr.handler.component.ShardFieldSortedHitQueue$1.compare(ShardFieldSortedHitQueue.java:153)
> at
> org.apache.solr.handler.component.ShardFieldSortedHitQueue.lessThan(ShardFieldSortedHitQueue.java:91)
> at
> org.apache.solr.handler.component.ShardFieldSortedHitQueue.lessThan(ShardFieldSortedHitQueue.java:33)
> at org.apache.lucene.util.PriorityQueue.upHeap(PriorityQueue.java:263)
> at org.apache.lucene.util.PriorityQueue.add(PriorityQueue.java:140)
> at
> org.apache.lucene.util.PriorityQueue.insertWithOverflow(PriorityQueue.java:156)
> at
> org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:924)
> at
> org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:585)
> at
> org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:564)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:423)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2484)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:720)
> at
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:526){code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12457) field(x,min|max) sorting doesn't work on trie or str fields in multi-shard collections
[
https://issues.apache.org/jira/browse/SOLR-12457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16510308#comment-16510308
]
Hoss Man commented on SOLR-12457:
-
bq. ... so StrField's marshalSortValue & unmarshalSortValue methods (which
handle the BytesRef/String conversion) take care of things for you.
FWIW: i started second guessing myself as to *why* I was seeing the implicit
syntax work for str fields, because in both cases the {{SortField}} should be
identical {{SortedSetSortField}} instances with the field name set on them --
but then i realized {{QueryComponent.unmarshalSortValues}} uses
{{SortSpec.getSchemaFields()}} to decide which {{SortFields}} should be
unmarshalled -- and _that_ knows the difference between a {{SortField}} whose
{{String getField()}} might happen to match a schema field name, vs an actual
usage of sorting on a {{SchemaField}} (which is important for field aliasing,
etc...)
> field(x,min|max) sorting doesn't work on trie or str fields in multi-shard
> collections
> --
>
> Key: SOLR-12457
> URL: https://issues.apache.org/jira/browse/SOLR-12457
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 7.1
>Reporter: Varun Thacker
>Priority: Major
>
> When I go to sort on a multi-valued field in a 2 shard collection, which has
> trie fields the query fails.
> To reproduce we need 2+ shards, a multi-valued trie field and "desc" sort
> criteria.
> Here's my schema
> {code:java}
> multiValued="true" docValues="true"/>
> positionIncrementGap="0" precisionStep="0"/>
> multiValued="true"/>
>
> {code}
> Now If I add a few docs
> {code:java}
> [
> {"id" : "1", "test_is" : ["1", "2", "3", "4", "5"], "test_i" : ["1", "2",
> "3", "4", "5"]},
> {"id" : "2", "test_is" : ["1", "2", "3", "4", "5"], "test_i" : ["1", "2",
> "3", "4", "5"]},
> {"id" : "3", "test_is" : ["1", "2", "3", "4", "5"], "test_i" : ["1", "2",
> "3", "4", "5"]}
> ]{code}
> Works:
> [http://localhost:8983/solr/gettingstarted/select?q=*:*=field(test_i,max)%20desc]
>
> Doesn't Work:
> [http://localhost:8983/solr/gettingstarted/select?q=*:*=field(test_is,max)%20desc]
>
> To be more clear when I say it doesn't work , the query throws and error and
> here's the stack trace for it:
> {code:java}
> ERROR - 2018-06-06 22:55:06.599; [c:gettingstarted s:shard2 r:core_node8
> x:gettingstarted_shard2_replica_n5] org.apache.solr.common.SolrException;
> null:java.lang.ClassCastException: java.lang.String cannot be cast to
> org.apache.lucene.util.BytesRef
> at
> org.apache.lucene.search.FieldComparator$TermOrdValComparator.compareValues(FieldComparator.java:561)
> at
> org.apache.solr.handler.component.ShardFieldSortedHitQueue$1.compare(ShardFieldSortedHitQueue.java:161)
> at
> org.apache.solr.handler.component.ShardFieldSortedHitQueue$1.compare(ShardFieldSortedHitQueue.java:153)
> at
> org.apache.solr.handler.component.ShardFieldSortedHitQueue.lessThan(ShardFieldSortedHitQueue.java:91)
> at
> org.apache.solr.handler.component.ShardFieldSortedHitQueue.lessThan(ShardFieldSortedHitQueue.java:33)
> at org.apache.lucene.util.PriorityQueue.upHeap(PriorityQueue.java:263)
> at org.apache.lucene.util.PriorityQueue.add(PriorityQueue.java:140)
> at
> org.apache.lucene.util.PriorityQueue.insertWithOverflow(PriorityQueue.java:156)
> at
> org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:924)
> at
> org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:585)
> at
> org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:564)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:423)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2484)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:720)
> at
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:526){code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
