Re: Weird query issues
Hello Shawn,
We found that it is unrelated to the group queries instead more
related to the empty queries. Do you happen to know what could cause empty
queries like the following from SOLRJ ? I can generate similar query via
curl hitting the select handler like - http://server:port/solr/select
server.log_2013-04-26T05-02-22:[#|2013-04-26T04:33:39.065-0400|INFO|sun-appserver2.1.1|org.apache.solr.core.SolrCore|_ThreadID=38;_ThreadName=httpSSLWorkerTh
read-9001-11;|[xxxcore] webapp=/solr path=/select params={} hits=24099
status=0 QTime=19 |#]
What we are seeing is a huge number of these empty queries. Once this
happens I have observed 2 things
1. even if I query from admin console, irrespective of the query, I get
same results as if its a cached page of *:* query. i.e. I cannot see the
query I entered in the server log, the query doesn't even come to the
server but I get same results as *:*
2. If I query via solrj no results are returned.
This has been driving me nuts for almost a week. Any help is greatly
appreciated.
Thanks
Ravi Kiran Bhaskar
On Sat, Apr 20, 2013 at 10:33 PM, Ravi Solr ravis...@gmail.com wrote:
Thanks for your advise Shawn. I have created a JIRA issue SOLR-4743.
On Sat, Apr 20, 2013 at 4:32 PM, Shawn Heisey s...@elyograg.org wrote:
On 4/20/2013 9:08 AM, Ravi Solr wrote:
Thanks you very much for responding Shawn. I never use IE, I use
firefox.
These are brand new servers and I don't think I am mixing versions. What
made you think I was using the 1.4.1 ?? You are correct in saying that
the
server is throwing HTML response since a group query has been failing
with
SEVERE error following which the entire instance behaves weirdly until
we
restart.
Its surprising that group query error handling has such glaring issue.
If
you specify group=true but don't specify group.query or group.field SOLR
throws a SEVERE exception following which we see the empty queries and
finally no responses via solrj and admin console gives numFound always
equal to total number of docs in index . Looks like the searcher goes
for a
spin once it encounters the exception. Such situation should have been
gracefully handled
Ah, so what's happening is that after an invalid grouping query, Solr is
unstable and stops working right. You should file an issue in Jira,
giving as much detail as you can. My last message was almost completely
wrong.
You are right that it should be gracefully handled, and obviously it is
not. For the 3.x Solr versions, grouping did not exist before 3.6. It
is a major 4.x feature that was backported. Sometimes such major
features depend on significant changes that have not happened on older
versions, leading to problems like this. Unfortunately, you could wait
quite a while for a fix on 3.6, where active development has stopped.
I have no personal experience with grouping, but I just tried the
problematic query (adding group=true to one that works) on 4.2.1. It
doesn't throw an error, I just get no results. When I follow it with a
regular query, everything works perfectly. Would you be able to upgrade
to 4.2.1? That's not a trivial thing to do, so hopefully you are
already working on upgrading.
Thanks,
Shawn
Re: Weird query issues
On 4/26/2013 1:01 PM, Ravi Solr wrote:
Hello Shawn,
We found that it is unrelated to the group queries instead more
related to the empty queries. Do you happen to know what could cause empty
queries like the following from SOLRJ ? I can generate similar query via
curl hitting the select handler like - http://server:port/solr/select
server.log_2013-04-26T05-02-22:[#|2013-04-26T04:33:39.065-0400|INFO|sun-appserver2.1.1|org.apache.solr.core.SolrCore|_ThreadID=38;_ThreadName=httpSSLWorkerTh
read-9001-11;|[xxxcore] webapp=/solr path=/select params={} hits=24099
status=0 QTime=19 |#]
What we are seeing is a huge number of these empty queries. Once this
happens I have observed 2 things
1. even if I query from admin console, irrespective of the query, I get
same results as if its a cached page of *:* query. i.e. I cannot see the
query I entered in the server log, the query doesn't even come to the
server but I get same results as *:*
2. If I query via solrj no results are returned.
This has been driving me nuts for almost a week. Any help is greatly
appreciated.
Querying from the admin UI and not seeing anything in the server log
sounds like browser caching. You can turn that off in solrconfig.xml.
I could not duplicate what you're seeing with SolrJ. You didn't say
what version of SolrJ, so I did this using 3.6.2 (same as your server
version). I thought maybe if you had a query object that didn't have an
actual query set, it might do what you're seeing, but that doesn't
appear to be the case. I don't have a 3.6.2 server to test against, so
I used my 3.5.0 and 4.2.1 servers.
Test code:
http://pastie.org/private/bnvurz1f9b9viawgqbxvmq
Solr 4.2.1 log:
INFO - 2013-04-26 14:17:24.127; org.apache.solr.core.SolrCore; [ncmain]
webapp=/solr path=/select params={wt=xmlversion=2.2} hits=0 status=0
QTime=20
3.5.0 server log:
Apr 26, 2013 2:20:23 PM org.apache.solr.common.SolrException log
SEVERE: java.lang.NullPointerException
Apr 26, 2013 2:20:23 PM org.apache.solr.core.SolrCore execute
INFO: [ncmain] webapp=/solr path=/select params={wt=xmlversion=2.2}
status=500 QTime=0
Apr 26, 2013 2:20:23 PM org.apache.solr.common.SolrException log
SEVERE: java.lang.NullPointerException
Same code without the setParser line:
Solr 4.2.1 log:
INFO - 2013-04-26 14:14:01.270; org.apache.solr.core.SolrCore; [ncmain]
webapp=/solr path=/select params={wt=javabinversion=2} hits=0 status=0
QTime=187
Thanks,
Shawn
Re: Weird query issues
Thanks Shawn, We are using 3.6.2 client and server. I cleared my browser
cache several times while querying (is that similar to clear cache in
solrconfig.xml ?). The query is logged in the solrj based client's
application container however I see it empty in the solr's application
container...so somehow it is getting swallowed by solr...Iam not able to
figure out how and why ?
Thanks
Ravi Kiran Bhaskar
On Fri, Apr 26, 2013 at 4:33 PM, Shawn Heisey s...@elyograg.org wrote:
On 4/26/2013 1:01 PM, Ravi Solr wrote:
Hello Shawn,
We found that it is unrelated to the group queries instead more
related to the empty queries. Do you happen to know what could cause
empty
queries like the following from SOLRJ ? I can generate similar query via
curl hitting the select handler like - http://server:port/solr/select
server.log_2013-04-26T05-02-22:[#|2013-04-26T04:33:39.065-0400|INFO|sun-appserver2.1.1|org.apache.solr.core.SolrCore|_ThreadID=38;_ThreadName=httpSSLWorkerTh
read-9001-11;|[xxxcore] webapp=/solr path=/select params={} hits=24099
status=0 QTime=19 |#]
What we are seeing is a huge number of these empty queries. Once this
happens I have observed 2 things
1. even if I query from admin console, irrespective of the query, I get
same results as if its a cached page of *:* query. i.e. I cannot see the
query I entered in the server log, the query doesn't even come to the
server but I get same results as *:*
2. If I query via solrj no results are returned.
This has been driving me nuts for almost a week. Any help is greatly
appreciated.
Querying from the admin UI and not seeing anything in the server log
sounds like browser caching. You can turn that off in solrconfig.xml.
I could not duplicate what you're seeing with SolrJ. You didn't say
what version of SolrJ, so I did this using 3.6.2 (same as your server
version). I thought maybe if you had a query object that didn't have an
actual query set, it might do what you're seeing, but that doesn't
appear to be the case. I don't have a 3.6.2 server to test against, so
I used my 3.5.0 and 4.2.1 servers.
Test code:
http://pastie.org/private/bnvurz1f9b9viawgqbxvmq
Solr 4.2.1 log:
INFO - 2013-04-26 14:17:24.127; org.apache.solr.core.SolrCore; [ncmain]
webapp=/solr path=/select params={wt=xmlversion=2.2} hits=0 status=0
QTime=20
3.5.0 server log:
Apr 26, 2013 2:20:23 PM org.apache.solr.common.SolrException log
SEVERE: java.lang.NullPointerException
Apr 26, 2013 2:20:23 PM org.apache.solr.core.SolrCore execute
INFO: [ncmain] webapp=/solr path=/select params={wt=xmlversion=2.2}
status=500 QTime=0
Apr 26, 2013 2:20:23 PM org.apache.solr.common.SolrException log
SEVERE: java.lang.NullPointerException
Same code without the setParser line:
Solr 4.2.1 log:
INFO - 2013-04-26 14:14:01.270; org.apache.solr.core.SolrCore; [ncmain]
webapp=/solr path=/select params={wt=javabinversion=2} hits=0 status=0
QTime=187
Thanks,
Shawn
Re: Weird query issues
Thanks you very much for responding Shawn. I never use IE, I use firefox.
These are brand new servers and I don't think I am mixing versions. What
made you think I was using the 1.4.1 ?? You are correct in saying that the
server is throwing HTML response since a group query has been failing with
SEVERE error following which the entire instance behaves weirdly until we
restart.
Its surprising that group query error handling has such glaring issue. If
you specify group=true but don't specify group.query or group.field SOLR
throws a SEVERE exception following which we see the empty queries and
finally no responses via solrj and admin console gives numFound always
equal to total number of docs in index . Looks like the searcher goes for a
spin once it encounters the exception. Such situation should have been
gracefully handled
[#|2013-04-19T23:47:53.363-0400|SEVERE|sun-appserver2.1.1|org.apache.solr.core.SolrCore|_ThreadID=26;_ThreadName=httpSSLWorkerThread-9001-17;_RequestID=2f
933642-cad0-40e5-86c6-65b00be9bb97;|org.apache.solr.common.SolrException:
Specify at least one field, function or query to group by.
at org.apache.solr.search.Grouping.execute(Grouping.java:228)
at
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:372)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:186)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:365)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:260)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:246)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:313)
at
org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:287)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:218)
at
org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:648)
at
org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:593)
at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:94)
at
com.sun.enterprise.web.PESessionLockingStandardPipeline.invoke(PESessionLockingStandardPipeline.java:98)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:222)
at
org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:648)
at
org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:593)
at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:587)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:1093)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:166)
at
org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:648)
at
org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:593)
at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:587)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:1093)
at org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:291)
at
com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.invokeAdapter(DefaultProcessorTask.java:670)
at
com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.doProcess(DefaultProcessorTask.java:601)
at
com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.process(DefaultProcessorTask.java:875)
at
com.sun.enterprise.web.connector.grizzly.DefaultReadTask.executeProcessorTask(DefaultReadTask.java:365)
at
com.sun.enterprise.web.connector.grizzly.DefaultReadTask.doTask(DefaultReadTask.java:285)
at
com.sun.enterprise.web.connector.grizzly.DefaultReadTask.doTask(DefaultReadTask.java:221)
at com.sun.enterprise.web.connector.grizzly.TaskBase.run(TaskBase.java:269)
at
com.sun.enterprise.web.connector.grizzly.ssl.SSLWorkerThread.run(SSLWorkerThread.java:111)
|#]
[#|2013-04-19T23:47:53.365-0400|INFO|sun-appserver2.1.1|org.apache.solr.core.SolrCore|_ThreadID=26;_ThreadName=httpSSLWorkerThread-9001-17;|[core1]
webapp=/solr path=/select
params={q=astronomy\+rows=10start=0facet=truefq=source:xxx.comfq=locations:(Maryland)sort=score+descgroup=true}
status=400 QTime=9 |#]
Ravi Kiran Bhaskar
On Fri, Apr 19, 2013 at 5:40 PM, Shawn Heisey s...@elyograg.org wrote:
On 4/19/2013 12:55 PM, Ravi Solr wrote:
We are using Solr 3.6.2 single core ( both index and query on same
machine)
and randomly the server fails to query correctly. If we query from the
admin console the query is not even applied and it returns numFound count
equal to total docs in the index as if no query is made, and if use SOLRJ
to query it throws javabin error
Invalid version (expected 2, but 60) or the data in not in 'javabin'
Re: Weird query issues
On 4/20/2013 9:08 AM, Ravi Solr wrote: Thanks you very much for responding Shawn. I never use IE, I use firefox. These are brand new servers and I don't think I am mixing versions. What made you think I was using the 1.4.1 ?? You are correct in saying that the server is throwing HTML response since a group query has been failing with SEVERE error following which the entire instance behaves weirdly until we restart. Its surprising that group query error handling has such glaring issue. If you specify group=true but don't specify group.query or group.field SOLR throws a SEVERE exception following which we see the empty queries and finally no responses via solrj and admin console gives numFound always equal to total number of docs in index . Looks like the searcher goes for a spin once it encounters the exception. Such situation should have been gracefully handled Ah, so what's happening is that after an invalid grouping query, Solr is unstable and stops working right. You should file an issue in Jira, giving as much detail as you can. My last message was almost completely wrong. You are right that it should be gracefully handled, and obviously it is not. For the 3.x Solr versions, grouping did not exist before 3.6. It is a major 4.x feature that was backported. Sometimes such major features depend on significant changes that have not happened on older versions, leading to problems like this. Unfortunately, you could wait quite a while for a fix on 3.6, where active development has stopped. I have no personal experience with grouping, but I just tried the problematic query (adding group=true to one that works) on 4.2.1. It doesn't throw an error, I just get no results. When I follow it with a regular query, everything works perfectly. Would you be able to upgrade to 4.2.1? That's not a trivial thing to do, so hopefully you are already working on upgrading. Thanks, Shawn
Re: Weird query issues
Thanks for your advise Shawn. I have created a JIRA issue SOLR-4743. On Sat, Apr 20, 2013 at 4:32 PM, Shawn Heisey s...@elyograg.org wrote: On 4/20/2013 9:08 AM, Ravi Solr wrote: Thanks you very much for responding Shawn. I never use IE, I use firefox. These are brand new servers and I don't think I am mixing versions. What made you think I was using the 1.4.1 ?? You are correct in saying that the server is throwing HTML response since a group query has been failing with SEVERE error following which the entire instance behaves weirdly until we restart. Its surprising that group query error handling has such glaring issue. If you specify group=true but don't specify group.query or group.field SOLR throws a SEVERE exception following which we see the empty queries and finally no responses via solrj and admin console gives numFound always equal to total number of docs in index . Looks like the searcher goes for a spin once it encounters the exception. Such situation should have been gracefully handled Ah, so what's happening is that after an invalid grouping query, Solr is unstable and stops working right. You should file an issue in Jira, giving as much detail as you can. My last message was almost completely wrong. You are right that it should be gracefully handled, and obviously it is not. For the 3.x Solr versions, grouping did not exist before 3.6. It is a major 4.x feature that was backported. Sometimes such major features depend on significant changes that have not happened on older versions, leading to problems like this. Unfortunately, you could wait quite a while for a fix on 3.6, where active development has stopped. I have no personal experience with grouping, but I just tried the problematic query (adding group=true to one that works) on 4.2.1. It doesn't throw an error, I just get no results. When I follow it with a regular query, everything works perfectly. Would you be able to upgrade to 4.2.1? That's not a trivial thing to do, so hopefully you are already working on upgrading. Thanks, Shawn
Weird query issues
Hello,
We are using Solr 3.6.2 single core ( both index and query on same machine)
and randomly the server fails to query correctly. If we query from the
admin console the query is not even applied and it returns numFound count
equal to total docs in the index as if no query is made, and if use SOLRJ
to query it throws javabin error
Invalid version (expected 2, but 60) or the data in not in 'javabin' format
Once we restart the container everything is back to normal.
In the process of debugging the solr logs I found empty queries like the
one below. Can anybody tell me what can cause empty queries in the log as
given below so trying to see if it may be relateed to the solr issues
[#|2013-04-19T14:10:20.308-0400|INFO|sun-appserver2.1.1|org.apache.solr.core.SolrCore|_ThreadID=19;_ThreadName=httpSSLWorkerThread-9001-0;|[core1]
webapp=/solr path=/select params={} hits=21727 status=0 QTime=24 |#]
Would Appreciate any pointers
Thanks
Ravi Kiran Bhaskar
Weird query issues
Hello,
We are using Solr 3.6.2 single core ( both index and query on same machine)
and randomly the server fails to query correctly. If we query from the
admin console the query is not even applied and it returns numFound count
equal to total docs in the index as if no query is made, and if use SOLRJ
to query it throws javabin error
Invalid version (expected 2, but 60) or the data in not in 'javabin' format
Once we restart the container everything is back to normal.
In the process of debugging the solr logs I found empty queries like the
one below. Can anybody tell me what can cause empty queries in the log as
given below so trying to see if it may be relateed to the solr issues
[#|2013-04-19T14:10:20.308-0400|INFO|sun-appserver2.1.1|org.apache.solr.core.SolrCore|_ThreadID=19;_ThreadName=httpSSLWorkerThread-9001-0;|[core1]
webapp=/solr path=/select params={} hits=21727 status=0 QTime=24 |#]
Would Appreciate any pointers
Thanks
Ravi Kiran Bhaskar
Re: Weird query issues
On 4/19/2013 12:55 PM, Ravi Solr wrote: We are using Solr 3.6.2 single core ( both index and query on same machine) and randomly the server fails to query correctly. If we query from the admin console the query is not even applied and it returns numFound count equal to total docs in the index as if no query is made, and if use SOLRJ to query it throws javabin error Invalid version (expected 2, but 60) or the data in not in 'javabin' format The UI problem is likely a browser issue, but I could be wrong. Some browsers, IE in particular, but not limited to that one, have problems with the admin UI. Using a different browser or clearing the browser cache can sometimes fix those problems. As for SolrJ, are you using a really old (1.x) SolrJ with Solr 3.6.2? Have you ever had Solr 1.x running on the same machine that's now running 3.6.2? Because the javabin version changed between 1.4.1 and 3.1.0, SolrJ 1.x is not compatible with Solr 3.1 and later unless you set the response parser on the server object to XML before you try to use it. If you have upgraded Solr from an old version, your servlet container (sun-appserver) may have some of the old jars remaining from the 1.x install. They must be removed. To change your SolrJ to use the XML response parser, use code like the following: server.setParser(new XMLResponseParser()); When SolrJ and Solr are both version 3.x or 4.x, you can remove this line. Another way that you can get the javabin error is when Solr is returning an error response, or returning a response that is not an error but is an HTML response reporting an unusual circumstance rather than the usual javabin. These HTML responses should no longer exist in the newest versions of Solr. Do you see any errors or warnings in your server log? The server log line you included in your email is not an error. Thanks, Shawn
