Thanks for the stack trace Greg, I cloned SOLR-39 into SOLR-40 to track this aspect of the problem.
: Date: Tue, 25 Jul 2006 21:39:19 -0700 : From: Greg Ludington <[EMAIL PROTECTED]> : Reply-To: [email protected] : To: [email protected] : Subject: Re: [jira] Resolved: (SOLR-39) Searcher's getDocListAndSet : methods do not accept flags, can cause NPE when writing output : : It may be a somewhat obscure pathway to produce this -- I only came : across it when, in applying faceting, I was using getDocListAndSet to : return both the DocList for output and the DocSet for facet : calculations, without fetching documents in any other way. Scores are : set to null here -- but, as you indicate, they are also set to null if : you getDocListNC, but that does not end up with an error. I agree : that the underlying issue should also be addressed, as well, but I : have not dug deeply enough into the internals to see the cause yet. : : Here is the stack trace, when using a getDocListAndSet method without flags: : : 09> Started [EMAIL PROTECTED] : Jul 25, 2006 9:32:22 PM org.apache.solr.core.SolrCore execute : INFO: rows=10&explainOther=&start=0&indent=on&q=dell&fl=&qt=dismax&stylesheet=&v : ersion=2.1 0 140 : Jul 25, 2006 9:32:22 PM org.apache.solr.core.SolrException log : SEVERE: java.lang.NullPointerException : at org.apache.solr.search.DocSlice$1.score(DocSlice.java:116) : at org.apache.solr.request.XMLWriter.writeDocList(XMLWriter.java:346) : at org.apache.solr.request.XMLWriter.writeVal(XMLWriter.java:385) : at org.apache.solr.request.XMLWriter.writeResponse(XMLWriter.java:106) : at org.apache.solr.request.XMLResponseWriter.write(XMLResponseWriter.jav : a:29) : at org.apache.solr.servlet.SolrServlet.doGet(SolrServlet.java:96) : at javax.servlet.http.HttpServlet.service(HttpServlet.java:596) : at javax.servlet.http.HttpServlet.service(HttpServlet.java:689) : at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:428 : ) : at org.mortbay.jetty.servlet.WebApplicationHandler.dispatch(WebApplicati : onHandler.java:473) : at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:5 : 68) : at org.mortbay.http.HttpContext.handle(HttpContext.java:1530) : at org.mortbay.jetty.servlet.WebApplicationContext.handle(WebApplication : Context.java:633) : at org.mortbay.http.HttpContext.handle(HttpContext.java:1482) : at org.mortbay.http.HttpServer.service(HttpServer.java:909) : at org.mortbay.http.HttpConnection.service(HttpConnection.java:820) : at org.mortbay.http.HttpConnection.handleNext(HttpConnection.java:986) : at org.mortbay.http.HttpConnection.handle(HttpConnection.java:837) : at org.mortbay.http.SocketListener.handleConnection(SocketListener.java: : 245) : at org.mortbay.util.ThreadedServer.handle(ThreadedServer.java:357) : at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:534) : : : Thanks, : Greg : : On 7/25/06, Chris Hostetter <[EMAIL PROTECTED]> wrote: : > : > : Thanks Greg, I just committed this. : > : > I'm all in favor of the patch as commited, but the NPE still concerens me : > ... the OutputWriter should be able to cleanly deal with a DocList that : > doesn't contain scores right? : > : > Should we open a seperate issue to look into this? ... it seems like it : > must be a somewhat obscure code path since I've certainly used Solr : > without scores in the past. : > : > Greg: do you by any chance have a stacktrace so we can see exactly where : > the NPE was getting thrown from? : > : > Yonik: do you have any idea what code path might cause an NPE? : > : > : : > : > Searcher's getDocListAndSet methods do not accept flags, can cause NPE when writing output : > : > ------------------------------------------------------------------------------------------ : > : > : > : > Key: SOLR-39 : > : > URL: http://issues.apache.org/jira/browse/SOLR-39 : > : > Project: Solr : > : > Issue Type: Bug : > : > Components: search : > : > Reporter: Greg Ludington : > : > Assigned To: Yonik Seeley : > : > Priority: Minor : > : > Attachments: SolrIndexSearcherdocListAndSet.patch : > : > : > : > : > : > SolrIndexSearcher's getDocListAndSet methods do not accept flags, which can, in some cases, cause a Null Pointer Exception to be thrown when writing the docListAndSet.docList as output. I came across the issue as I was implementing faceting, see http://www.nabble.com/Faceted-Browsing-Question-Discussion-tf1968854.html for the discussion. : > : > The simplest way to reproduce this is to modify DisMaxRequestHandler, by changing this: : > : > DocList results = s.getDocList(query, restrictions, : > : > SolrPluginUtils.getSort(req), : > : > req.getStart(), req.getLimit(), : > : > flags); : > : > rsp.add("search-results",results); : > : > to : > : > DocListAndSet listAndSet= s.getDocListAndSet(query, restrictions, : > : > SolrPluginUtils.getSort(req), : > : > req.getStart(), req.getLimit()); : > : > DocList results = listAndSet.docList; : > : > rsp.add("search-results",results); : > : > The root cause appears to be that the scores[] is set to null, so then the DocIterator and its score() method is called, return scores[pos-1] will give null. When getDocListAndSet(..) is invoked, it eventually can get down to this private method: : > : > private DocSet getDocListAndSetNC(DocListAndSet out, Query query, DocSet filter, Sort lsort, int offset, int len, int flags) throws IOException : > : > In that method, scores is assigned as follows: : > : > scores = (flags&GET_SCORES)!=0 ? new float[nDocsReturned] : null; : > : > Since getDocListAndSet() does not pass flags (except for the implicit GET_DOCSET), scores is assigned as null, which eventually leads to the NullPointerException if you try to output the docList . The attached patch does not change the underlying mechanism of how scores is assigned, but works around the issue by adding overloaded getDocListAndSet() methods that take an additional flags parameter. After applying this patch, you can change the relevant bit in DisMaxRequestHandler to: : > : > DocListAndSet listAndSet= s.getDocListAndSet(query, restrictions, : > : > SolrPluginUtils.getSort(req), : > : > req.getStart(), req.getLimit(), flags); : > : > DocList results = listAndSet.docList; : > : > rsp.add("search-results",results); : > : > and you will no longer see the NullPointerException : > : : > : -- : > : This message is automatically generated by JIRA. : > : - : > : If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa : > : - : > : For more information on JIRA, see: http://www.atlassian.com/software/jira : > : : > : : > : : > : > : > : > -Hoss : > : > : -Hoss
