Re: Distance sorting with spatial filtering
I tracked down the problem and found a workaround. If there is a wildcard entry in schema.xml such as the following. !-- Ignore any fields that don't already match an existing field name -- dynamicField name=* type=ignored multiValued=true / then sort by function fails and returns Error 400 can not sort on unindexed field: function name Removing the name=* entry from schema.xml is a workaround. I noted this in the Solr-1297 JIRA entry. Scott On Fri, Sep 10, 2010 at 01:40, Lance Norskog goks...@gmail.com wrote: Since no one has jumped in to give the right syntax- yeah, it's a bug. Please file a JIRA. On Thu, Sep 9, 2010 at 9:44 PM, Scott K s...@skister.com wrote: On Thu, Sep 9, 2010 at 21:00, Lance Norskog goks...@gmail.com wrote: I just checked out the trunk, and branch 3.x This query is accepted on both, but gives no responses: http://localhost:8983/solr/select/?q=*:*sort=dist(2,x_dt,y_dt,0,0)+asc So you are saying when you add the sort parameter you get no results back, but do not get the error I am seeing? Should I open a Jira ticket? x_dt and y_dt are wildcard fields with the tdouble type. tdouble explicitly says it is stored and indexed. Your 'longitude' and 'latitude' fields may not be stored? No, they are stored. http://localhost:8983/solr/select?q=*:*rows=1wt=xmlindent=true ?xml version=1.0 encoding=UTF-8? response lst name=responseHeader int name=status0/int int name=QTime9/int /lst result name=response numFound=365775 start=0 doc ... double name=latitude47.6636/double double name=longitude-122.3054/double Also, this is accepted on both branches: http://localhost:8983/solr/select/?q=*:*sort=sum(1)+asc The documentation for sum() does not mention single-argument calls. This also fails http://localhost:8983/solr/select/?q=*:*sort=sum(1,2)+asc http://localhost:8983/solr/select/?q=*:*sort=sum(latitude,longitude)+asc Scott K wrote: According to the documentation, sorting by function has been a feature since Solr 1.5. It seems like a major regression if this no longer works. http://wiki.apache.org/solr/FunctionQuery#Sort_By_Function The _val_ trick does not seem to work if used with a query term, although I can try some more things to give 0 value to the query term. On Wed, Sep 8, 2010 at 22:21, Lance Norskoggoks...@gmail.com wrote: It says that the field sum(1) is not indexed. You don't have a field called 'sum(1)'. I know there has been a lot of changes in query parsing, and sorting by functions may be on the list. But the _val_ trick is the older one and, and you noted, still works. The _val_ trick sets the ranking value to the output of the function, thus indirectly doing what sort= does. Lance Scott K wrote: I get the error on all functions. GET 'http://localhost:8983/solr/select?q=*:*sort=sum(1)+asc' Error 400 can not sort on unindexed field: sum(1) I tried another nightly build from today, Sep 7th, with the same results. I attached the schema.xml Thanks for the help! Scott On Wed, Sep 1, 2010 at 18:43, Lance Norskoggoks...@gmail.com wrote: Post your schema. On Mon, Aug 30, 2010 at 2:04 PM, Scott Ks...@skister.com wrote: The new spatial filtering (SOLR-1586) works great and is much faster than fq={!frange. However, I am having problems sorting by distance. If I try GET 'http://localhost:8983/solr/select/?q=*:*sort=dist(2,latitude,longitude,0,0)+asc' I get an error: Error 400 can not sort on unindexed field: dist(2,latitude,longitude,0,0) I was able to work around this with GET 'http://localhost:8983/solr/select/?q=*:* AND _val_:recip(dist(2, latitude, longitude, 0,0),1,1,1)fl=*,score' But why isn't sorting by functions working? I get this error with any function I try to sort on.This is a nightly trunk build from Aug 25th. I see SOLR-1297 was reopened, but that seems to be for edge cases. Second question: I am using the LatLonType from the Spatial Filtering wiki, http://wiki.apache.org/solr/SpatialSearch Are there any distance sorting functions that use this field, or do I need to have three indexed fields, store_lat_lon, latitude, and longitude, if I want both filtering and sorting by distance. Thanks, Scott -- Lance Norskog goks...@gmail.com -- Lance Norskog goks...@gmail.com
Re: Distance sorting with spatial filtering
Since no one has jumped in to give the right syntax- yeah, it's a bug. Please file a JIRA. On Thu, Sep 9, 2010 at 9:44 PM, Scott K s...@skister.com wrote: On Thu, Sep 9, 2010 at 21:00, Lance Norskog goks...@gmail.com wrote: I just checked out the trunk, and branch 3.x This query is accepted on both, but gives no responses: http://localhost:8983/solr/select/?q=*:*sort=dist(2,x_dt,y_dt,0,0)+asc So you are saying when you add the sort parameter you get no results back, but do not get the error I am seeing? Should I open a Jira ticket? x_dt and y_dt are wildcard fields with the tdouble type. tdouble explicitly says it is stored and indexed. Your 'longitude' and 'latitude' fields may not be stored? No, they are stored. http://localhost:8983/solr/select?q=*:*rows=1wt=xmlindent=true ?xml version=1.0 encoding=UTF-8? response lst name=responseHeader int name=status0/int int name=QTime9/int /lst result name=response numFound=365775 start=0 doc ... double name=latitude47.6636/double double name=longitude-122.3054/double Also, this is accepted on both branches: http://localhost:8983/solr/select/?q=*:*sort=sum(1)+asc The documentation for sum() does not mention single-argument calls. This also fails http://localhost:8983/solr/select/?q=*:*sort=sum(1,2)+asc http://localhost:8983/solr/select/?q=*:*sort=sum(latitude,longitude)+asc Scott K wrote: According to the documentation, sorting by function has been a feature since Solr 1.5. It seems like a major regression if this no longer works. http://wiki.apache.org/solr/FunctionQuery#Sort_By_Function The _val_ trick does not seem to work if used with a query term, although I can try some more things to give 0 value to the query term. On Wed, Sep 8, 2010 at 22:21, Lance Norskoggoks...@gmail.com wrote: It says that the field sum(1) is not indexed. You don't have a field called 'sum(1)'. I know there has been a lot of changes in query parsing, and sorting by functions may be on the list. But the _val_ trick is the older one and, and you noted, still works. The _val_ trick sets the ranking value to the output of the function, thus indirectly doing what sort= does. Lance Scott K wrote: I get the error on all functions. GET 'http://localhost:8983/solr/select?q=*:*sort=sum(1)+asc' Error 400 can not sort on unindexed field: sum(1) I tried another nightly build from today, Sep 7th, with the same results. I attached the schema.xml Thanks for the help! Scott On Wed, Sep 1, 2010 at 18:43, Lance Norskoggoks...@gmail.com wrote: Post your schema. On Mon, Aug 30, 2010 at 2:04 PM, Scott Ks...@skister.com wrote: The new spatial filtering (SOLR-1586) works great and is much faster than fq={!frange. However, I am having problems sorting by distance. If I try GET 'http://localhost:8983/solr/select/?q=*:*sort=dist(2,latitude,longitude,0,0)+asc' I get an error: Error 400 can not sort on unindexed field: dist(2,latitude,longitude,0,0) I was able to work around this with GET 'http://localhost:8983/solr/select/?q=*:* AND _val_:recip(dist(2, latitude, longitude, 0,0),1,1,1)fl=*,score' But why isn't sorting by functions working? I get this error with any function I try to sort on.This is a nightly trunk build from Aug 25th. I see SOLR-1297 was reopened, but that seems to be for edge cases. Second question: I am using the LatLonType from the Spatial Filtering wiki, http://wiki.apache.org/solr/SpatialSearch Are there any distance sorting functions that use this field, or do I need to have three indexed fields, store_lat_lon, latitude, and longitude, if I want both filtering and sorting by distance. Thanks, Scott -- Lance Norskog goks...@gmail.com -- Lance Norskog goks...@gmail.com
Re: Distance sorting with spatial filtering
According to the documentation, sorting by function has been a feature since Solr 1.5. It seems like a major regression if this no longer works. http://wiki.apache.org/solr/FunctionQuery#Sort_By_Function The _val_ trick does not seem to work if used with a query term, although I can try some more things to give 0 value to the query term. On Wed, Sep 8, 2010 at 22:21, Lance Norskog goks...@gmail.com wrote: It says that the field sum(1) is not indexed. You don't have a field called 'sum(1)'. I know there has been a lot of changes in query parsing, and sorting by functions may be on the list. But the _val_ trick is the older one and, and you noted, still works. The _val_ trick sets the ranking value to the output of the function, thus indirectly doing what sort= does. Lance Scott K wrote: I get the error on all functions. GET 'http://localhost:8983/solr/select?q=*:*sort=sum(1)+asc' Error 400 can not sort on unindexed field: sum(1) I tried another nightly build from today, Sep 7th, with the same results. I attached the schema.xml Thanks for the help! Scott On Wed, Sep 1, 2010 at 18:43, Lance Norskoggoks...@gmail.com wrote: Post your schema. On Mon, Aug 30, 2010 at 2:04 PM, Scott Ks...@skister.com wrote: The new spatial filtering (SOLR-1586) works great and is much faster than fq={!frange. However, I am having problems sorting by distance. If I try GET 'http://localhost:8983/solr/select/?q=*:*sort=dist(2,latitude,longitude,0,0)+asc' I get an error: Error 400 can not sort on unindexed field: dist(2,latitude,longitude,0,0) I was able to work around this with GET 'http://localhost:8983/solr/select/?q=*:* AND _val_:recip(dist(2, latitude, longitude, 0,0),1,1,1)fl=*,score' But why isn't sorting by functions working? I get this error with any function I try to sort on.This is a nightly trunk build from Aug 25th. I see SOLR-1297 was reopened, but that seems to be for edge cases. Second question: I am using the LatLonType from the Spatial Filtering wiki, http://wiki.apache.org/solr/SpatialSearch Are there any distance sorting functions that use this field, or do I need to have three indexed fields, store_lat_lon, latitude, and longitude, if I want both filtering and sorting by distance. Thanks, Scott -- Lance Norskog goks...@gmail.com
Re: Distance sorting with spatial filtering
I just checked out the trunk, and branch 3.x This query is accepted on both, but gives no responses: http://localhost:8983/solr/select/?q=*:*sort=dist(2,x_dt,y_dt,0,0)+asc x_dt and y_dt are wildcard fields with the tdouble type. tdouble explicitly says it is stored and indexed. Your 'longitude' and 'latitude' fields may not be stored? Also, this is accepted on both branches: http://localhost:8983/solr/select/?q=*:*sort=sum(1)+asc The documentation for sum() does not mention single-argument calls. (The name Solr 1.5 is dead- the next will probably be 3.1.) Scott K wrote: According to the documentation, sorting by function has been a feature since Solr 1.5. It seems like a major regression if this no longer works. http://wiki.apache.org/solr/FunctionQuery#Sort_By_Function The _val_ trick does not seem to work if used with a query term, although I can try some more things to give 0 value to the query term. On Wed, Sep 8, 2010 at 22:21, Lance Norskoggoks...@gmail.com wrote: It says that the field sum(1) is not indexed. You don't have a field called 'sum(1)'. I know there has been a lot of changes in query parsing, and sorting by functions may be on the list. But the _val_ trick is the older one and, and you noted, still works. The _val_ trick sets the ranking value to the output of the function, thus indirectly doing what sort= does. Lance Scott K wrote: I get the error on all functions. GET 'http://localhost:8983/solr/select?q=*:*sort=sum(1)+asc' Error 400 can not sort on unindexed field: sum(1) I tried another nightly build from today, Sep 7th, with the same results. I attached the schema.xml Thanks for the help! Scott On Wed, Sep 1, 2010 at 18:43, Lance Norskoggoks...@gmail.comwrote: Post your schema. On Mon, Aug 30, 2010 at 2:04 PM, Scott Ks...@skister.comwrote: The new spatial filtering (SOLR-1586) works great and is much faster than fq={!frange. However, I am having problems sorting by distance. If I try GET 'http://localhost:8983/solr/select/?q=*:*sort=dist(2,latitude,longitude,0,0)+asc' I get an error: Error 400 can not sort on unindexed field: dist(2,latitude,longitude,0,0) I was able to work around this with GET 'http://localhost:8983/solr/select/?q=*:* AND _val_:recip(dist(2, latitude, longitude, 0,0),1,1,1)fl=*,score' But why isn't sorting by functions working? I get this error with any function I try to sort on.This is a nightly trunk build from Aug 25th. I see SOLR-1297 was reopened, but that seems to be for edge cases. Second question: I am using the LatLonType from the Spatial Filtering wiki, http://wiki.apache.org/solr/SpatialSearch Are there any distance sorting functions that use this field, or do I need to have three indexed fields, store_lat_lon, latitude, and longitude, if I want both filtering and sorting by distance. Thanks, Scott -- Lance Norskog goks...@gmail.com
Re: Distance sorting with spatial filtering
On Thu, Sep 9, 2010 at 21:00, Lance Norskog goks...@gmail.com wrote: I just checked out the trunk, and branch 3.x This query is accepted on both, but gives no responses: http://localhost:8983/solr/select/?q=*:*sort=dist(2,x_dt,y_dt,0,0)+asc So you are saying when you add the sort parameter you get no results back, but do not get the error I am seeing? Should I open a Jira ticket? x_dt and y_dt are wildcard fields with the tdouble type. tdouble explicitly says it is stored and indexed. Your 'longitude' and 'latitude' fields may not be stored? No, they are stored. http://localhost:8983/solr/select?q=*:*rows=1wt=xmlindent=true ?xml version=1.0 encoding=UTF-8? response lst name=responseHeader int name=status0/int int name=QTime9/int /lst result name=response numFound=365775 start=0 doc ... double name=latitude47.6636/double double name=longitude-122.3054/double Also, this is accepted on both branches: http://localhost:8983/solr/select/?q=*:*sort=sum(1)+asc The documentation for sum() does not mention single-argument calls. This also fails http://localhost:8983/solr/select/?q=*:*sort=sum(1,2)+asc http://localhost:8983/solr/select/?q=*:*sort=sum(latitude,longitude)+asc Scott K wrote: According to the documentation, sorting by function has been a feature since Solr 1.5. It seems like a major regression if this no longer works. http://wiki.apache.org/solr/FunctionQuery#Sort_By_Function The _val_ trick does not seem to work if used with a query term, although I can try some more things to give 0 value to the query term. On Wed, Sep 8, 2010 at 22:21, Lance Norskoggoks...@gmail.com wrote: It says that the field sum(1) is not indexed. You don't have a field called 'sum(1)'. I know there has been a lot of changes in query parsing, and sorting by functions may be on the list. But the _val_ trick is the older one and, and you noted, still works. The _val_ trick sets the ranking value to the output of the function, thus indirectly doing what sort= does. Lance Scott K wrote: I get the error on all functions. GET 'http://localhost:8983/solr/select?q=*:*sort=sum(1)+asc' Error 400 can not sort on unindexed field: sum(1) I tried another nightly build from today, Sep 7th, with the same results. I attached the schema.xml Thanks for the help! Scott On Wed, Sep 1, 2010 at 18:43, Lance Norskoggoks...@gmail.com wrote: Post your schema. On Mon, Aug 30, 2010 at 2:04 PM, Scott Ks...@skister.com wrote: The new spatial filtering (SOLR-1586) works great and is much faster than fq={!frange. However, I am having problems sorting by distance. If I try GET 'http://localhost:8983/solr/select/?q=*:*sort=dist(2,latitude,longitude,0,0)+asc' I get an error: Error 400 can not sort on unindexed field: dist(2,latitude,longitude,0,0) I was able to work around this with GET 'http://localhost:8983/solr/select/?q=*:* AND _val_:recip(dist(2, latitude, longitude, 0,0),1,1,1)fl=*,score' But why isn't sorting by functions working? I get this error with any function I try to sort on.This is a nightly trunk build from Aug 25th. I see SOLR-1297 was reopened, but that seems to be for edge cases. Second question: I am using the LatLonType from the Spatial Filtering wiki, http://wiki.apache.org/solr/SpatialSearch Are there any distance sorting functions that use this field, or do I need to have three indexed fields, store_lat_lon, latitude, and longitude, if I want both filtering and sorting by distance. Thanks, Scott -- Lance Norskog goks...@gmail.com
Re: Distance sorting with spatial filtering
I get the error on all functions. GET 'http://localhost:8983/solr/select?q=*:*sort=sum(1)+asc' Error 400 can not sort on unindexed field: sum(1) I tried another nightly build from today, Sep 7th, with the same results. I attached the schema.xml Thanks for the help! Scott On Wed, Sep 1, 2010 at 18:43, Lance Norskog goks...@gmail.com wrote: Post your schema. On Mon, Aug 30, 2010 at 2:04 PM, Scott K s...@skister.com wrote: The new spatial filtering (SOLR-1586) works great and is much faster than fq={!frange. However, I am having problems sorting by distance. If I try GET 'http://localhost:8983/solr/select/?q=*:*sort=dist(2,latitude,longitude,0,0)+asc' I get an error: Error 400 can not sort on unindexed field: dist(2,latitude,longitude,0,0) I was able to work around this with GET 'http://localhost:8983/solr/select/?q=*:* AND _val_:recip(dist(2, latitude, longitude, 0,0),1,1,1)fl=*,score' But why isn't sorting by functions working? I get this error with any function I try to sort on.This is a nightly trunk build from Aug 25th. I see SOLR-1297 was reopened, but that seems to be for edge cases. Second question: I am using the LatLonType from the Spatial Filtering wiki, http://wiki.apache.org/solr/SpatialSearch Are there any distance sorting functions that use this field, or do I need to have three indexed fields, store_lat_lon, latitude, and longitude, if I want both filtering and sorting by distance. Thanks, Scott -- Lance Norskog goks...@gmail.com ?xml version=1.0 encoding=UTF-8 ? !-- PERFORMANCE NOTE: this schema includes many optional features and should not be used for benchmarking. To improve performance one could - set stored=false for all fields possible (esp large fields) when you only need to search on the field but don't need to return the original value. - set indexed=false if you don't need to search on the field, but only return the field as a result of searching on other indexed fields. - remove all unneeded copyField statements - for best index size and searching performance, set index to false for all general text fields, use copyField to copy them to the catchall text field, and use that for searching. - For maximum indexing performance, use the StreamingUpdateSolrServer java client. - Remember to run the JVM in server mode, and use a higher logging level that avoids logging every request -- schema name=schema version=1.2 types fieldType name=string class=solr.StrField sortMissingLast=true omitNorms=true/ !-- boolean type: true or false -- fieldType name=boolean class=solr.BoolField sortMissingLast=true omitNorms=true/ !--Binary data type. The data should be sent/retrieved in as Base64 encoded Strings -- fieldtype name=binary class=solr.BinaryField/ !-- Default numeric field types. For faster range queries, consider the tint/tfloat/tlong/tdouble types. -- fieldType name=int class=solr.TrieIntField precisionStep=0 omitNorms=true positionIncrementGap=0/ fieldType name=float class=solr.TrieFloatField precisionStep=0 omitNorms=true positionIncrementGap=0/ fieldType name=long class=solr.TrieLongField precisionStep=0 omitNorms=true positionIncrementGap=0/ fieldType name=double class=solr.TrieDoubleField precisionStep=0 omitNorms=true positionIncrementGap=0/ fieldType name=tint class=solr.TrieIntField precisionStep=8 omitNorms=true positionIncrementGap=0/ fieldType name=tfloat class=solr.TrieFloatField precisionStep=8 omitNorms=true positionIncrementGap=0/ fieldType name=tlong class=solr.TrieLongField precisionStep=8 omitNorms=true positionIncrementGap=0/ fieldType name=tdouble class=solr.TrieDoubleField precisionStep=8 omitNorms=true positionIncrementGap=0/ fieldType name=date class=solr.TrieDateField omitNorms=true precisionStep=0 positionIncrementGap=0/ !-- A Trie based date field for faster date range queries and date faceting. -- fieldType name=tdate class=solr.TrieDateField omitNorms=true precisionStep=6 positionIncrementGap=0/ fieldType name=random class=solr.RandomSortField indexed=true / fieldType name=location class=solr.LatLonType subFieldType=double / !-- A text field that only splits on whitespace for exact matching of words -- fieldType name=text_ws class=solr.TextField positionIncrementGap=100 analyzer tokenizer class=solr.WhitespaceTokenizerFactory/ /analyzer /fieldType fieldType name=text class=solr.TextField positionIncrementGap=100 analyzer type=index tokenizer class=solr.WhitespaceTokenizerFactory/ filter class=solr.StopFilterFactory ignoreCase=true words=stopwords.txt enablePositionIncrements=true / filter class=solr.WordDelimiterFilterFactory generateWordParts=1 generateNumberParts=1 catenateWords=1 catenateNumbers=1 catenateAll=0 splitOnCaseChange=1/ filter
Re: Distance sorting with spatial filtering
It says that the field sum(1) is not indexed. You don't have a field called 'sum(1)'. I know there has been a lot of changes in query parsing, and sorting by functions may be on the list. But the _val_ trick is the older one and, and you noted, still works. The _val_ trick sets the ranking value to the output of the function, thus indirectly doing what sort= does. Lance Scott K wrote: I get the error on all functions. GET 'http://localhost:8983/solr/select?q=*:*sort=sum(1)+asc' Error 400 can not sort on unindexed field: sum(1) I tried another nightly build from today, Sep 7th, with the same results. I attached the schema.xml Thanks for the help! Scott On Wed, Sep 1, 2010 at 18:43, Lance Norskoggoks...@gmail.com wrote: Post your schema. On Mon, Aug 30, 2010 at 2:04 PM, Scott Ks...@skister.com wrote: The new spatial filtering (SOLR-1586) works great and is much faster than fq={!frange. However, I am having problems sorting by distance. If I try GET 'http://localhost:8983/solr/select/?q=*:*sort=dist(2,latitude,longitude,0,0)+asc' I get an error: Error 400 can not sort on unindexed field: dist(2,latitude,longitude,0,0) I was able to work around this with GET 'http://localhost:8983/solr/select/?q=*:* AND _val_:recip(dist(2, latitude, longitude, 0,0),1,1,1)fl=*,score' But why isn't sorting by functions working? I get this error with any function I try to sort on.This is a nightly trunk build from Aug 25th. I see SOLR-1297 was reopened, but that seems to be for edge cases. Second question: I am using the LatLonType from the Spatial Filtering wiki, http://wiki.apache.org/solr/SpatialSearch Are there any distance sorting functions that use this field, or do I need to have three indexed fields, store_lat_lon, latitude, and longitude, if I want both filtering and sorting by distance. Thanks, Scott -- Lance Norskog goks...@gmail.com
Re: Distance sorting with spatial filtering
Post your schema. On Mon, Aug 30, 2010 at 2:04 PM, Scott K s...@skister.com wrote: The new spatial filtering (SOLR-1586) works great and is much faster than fq={!frange. However, I am having problems sorting by distance. If I try GET 'http://localhost:8983/solr/select/?q=*:*sort=dist(2,latitude,longitude,0,0)+asc' I get an error: Error 400 can not sort on unindexed field: dist(2,latitude,longitude,0,0) I was able to work around this with GET 'http://localhost:8983/solr/select/?q=*:* AND _val_:recip(dist(2, latitude, longitude, 0,0),1,1,1)fl=*,score' But why isn't sorting by functions working? I get this error with any function I try to sort on.This is a nightly trunk build from Aug 25th. I see SOLR-1297 was reopened, but that seems to be for edge cases. Second question: I am using the LatLonType from the Spatial Filtering wiki, http://wiki.apache.org/solr/SpatialSearch Are there any distance sorting functions that use this field, or do I need to have three indexed fields, store_lat_lon, latitude, and longitude, if I want both filtering and sorting by distance. Thanks, Scott -- Lance Norskog goks...@gmail.com
Distance sorting with spatial filtering
The new spatial filtering (SOLR-1586) works great and is much faster than fq={!frange. However, I am having problems sorting by distance. If I try GET 'http://localhost:8983/solr/select/?q=*:*sort=dist(2,latitude,longitude,0,0)+asc' I get an error: Error 400 can not sort on unindexed field: dist(2,latitude,longitude,0,0) I was able to work around this with GET 'http://localhost:8983/solr/select/?q=*:* AND _val_:recip(dist(2, latitude, longitude, 0,0),1,1,1)fl=*,score' But why isn't sorting by functions working? I get this error with any function I try to sort on.This is a nightly trunk build from Aug 25th. I see SOLR-1297 was reopened, but that seems to be for edge cases. Second question: I am using the LatLonType from the Spatial Filtering wiki, http://wiki.apache.org/solr/SpatialSearch Are there any distance sorting functions that use this field, or do I need to have three indexed fields, store_lat_lon, latitude, and longitude, if I want both filtering and sorting by distance. Thanks, Scott