[ 
https://issues.apache.org/jira/browse/SOLR-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14233501#comment-14233501
 ] 

Michael Ryan edited comment on SOLR-6469 at 12/3/14 8:48 PM:
-------------------------------------------------------------

We ran into this exception today, though for a slightly different reason. 
Simplified request looks like this:
q=\*:\*&fl=articleId&rows=1&hl=true&shards=foo1,foo2

The workaround in this case is to either request additional fields besides the 
unique key field, or to not request highlighting.

This hits the same NPE at HighlightComponent.java:189. It throws an NPE because 
more highlighting data is returned from the shards than are in resultIds. This 
happens because onePassDistributedQuery is set to true in QueryComponent, and 
so the highlight data is requested from the shards on the first (single) pass.

The NPE is easy enough to fix for my issue - just add a null check before sdoc 
is used, and skip it if null. But there is still an underlying inefficiency - 
many more docs are highlighted on the shards than are actually necessary. If 
you have a lot of shards, this highlighting becomes non-trivial (e.g., 
highlighting thousands of docs instead of hundreds). I don't see any way to fix 
this without doing a second pass. Maybe ignore onePassDistributedQuery if 
highlights are requested? (That'll make the distributed logic in 
QueryComponent/HighlightComponent even more fun.)

Should I create another Jira for my issue since it doesn't involve grouping?


was (Author: michaelryan):
We ran into this exception today, though for a slightly different reason. 
Request looks like this:
q=\*:\*&fl=articleId&rows=1&hl=true&shards=foo1,foo2

The workaround in this case is to either request additional fields besides the 
unique key field, or to not request highlighting in this case.

This hits the same NPE at HighlightComponent.java:189. It throws an NPE because 
more highlighting data is returned from the shards than are in resultIds. This 
happens because onePassDistributedQuery is set to true in QueryComponent, and 
so the highlight data is requested from the shards on the first (single) pass.

The NPE is easy enough to fix for my issue - just add a null check before sdoc 
is used, and skip it if null. But there is still an underlying inefficiency - 
many more docs are highlighted on the shards than are actually necessary. If 
you have a lot of shards, this highlighting becomes non-trivial (e.g., 
highlighting thousands of docs instead of hundreds). I don't see any way to fix 
this without doing a second pass. Maybe ignore onePassDistributedQuery if 
highlights are requested? (That'll make the distributed logic in 
QueryComponent/HighlightComponent even more fun.)

Should I create another Jira for my issue since it doesn't involve grouping?

> Solr search with multicore + grouping + highlighting cause NPE
> --------------------------------------------------------------
>
>                 Key: SOLR-6469
>                 URL: https://issues.apache.org/jira/browse/SOLR-6469
>             Project: Solr
>          Issue Type: Bug
>          Components: highlighter, multicore, SearchComponents - other
>    Affects Versions: 4.8.1
>         Environment: Windows 7, Intellij
>            Reporter: Shay Sofer
>            Assignee: Noble Paul
>              Labels: patch
>         Attachments: SOLR-6469.patch
>
>
> Integration of Grouping + shards + highlighting cause NullPointerException.
> Query: 
> localhost:8983/solr/Global_A/select?q=%2Btext%3A%28shay*%29+&rows=100&fl=id%2CobjId%2Cnull&shards=http%3A%2F%2F127.0.0.1%3A8983%2Fsolr%2F0_A%2Chttp%3A%2F%2F127.0.0.1%3A8983%2Fsolr%2FGlobal_A&group=true&group.query=name__s%3Ashay&sort=name__s_sort+asc&hl=true
> results:
> java.lang.NullPointerException
>  at 
> org.apache.solr.handler.component.HighlightComponent.finishStage(HighlightComponent.java:189)
>  at 
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:330)
>  at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
>  at org.apache.solr.core.SolrCore.execute(SolrCore.java:1952)
>  at 
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:774)
>  at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)
>  at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
>  at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
>  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
>  at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
>  at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
>  at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
>  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
>  at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
>  at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
>  at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
>  at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
>  at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
>  at org.eclipse.jetty.server.Server.handle(Server.java:368)
>  at 
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
>  at 
> org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
>  at 
> org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
>  at 
> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
>  at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
>  at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
>  at 
> org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
>  at 
> org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
>  at 
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
>  at 
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
>  at java.lang.Thread.run(Thread.java:722)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Reply via email to