[
https://issues.apache.org/jira/browse/SOLR-5426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14031042#comment-14031042
]
Uwe Schindler commented on SOLR-5426:
-------------------------------------
Hi,
the issue is in ReversedWildcardTokenFilter: The TokenFilter does not correctly
implement reset(). If the TokenStream is reused and it was not completely
consumed before, the state is still active. In that case it restores the "save"
state and so injects a buggy state from the previous usage as first token into
the new one.
The fix is to make ReverseWildcardTokenFilter correctly implement reset() and
NULL all state. The pseudo-fix in the highligters's tokenfilter just hides the
bug.
I will provide a patch in a minute!
> org.apache.solr.common.SolrException; org.apache.solr.common.SolrException:
> org.apache.lucene.search.highlight.InvalidTokenOffsetsException: Token 0
> exceeds length of provided text sized 840
> ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-5426
> URL: https://issues.apache.org/jira/browse/SOLR-5426
> Project: Solr
> Issue Type: Bug
> Components: highlighter
> Affects Versions: 4.4, 4.5.1
> Reporter: Nikolay
> Priority: Minor
> Attachments: SOLR-5426.patch, SOLR-5426.patch, SOLR-5426.patch,
> SOLR-5426.patch, SOLR-5426.patch, highlighter.zip
>
>
> Highlighter does not work correctly on test-data.
> I added index- and config- files (see attached highlighter.zip) for
> reproducing this issue.
> Everything works fine if I search without highlighting:
> http://localhost:8983/solr/global/select?q=aa&wt=json&indent=true
> But if search with highlighting:
> http://localhost:8983/solr/global/select?q=aa&wt=json&indent=true&hl=true&hl.fl=*_stx&hl.simple.pre=<em>&hl.simple.post=<%2Fem>
> I'm get the error:
> ERROR - 2013-11-07 10:17:15.797; org.apache.solr.common.SolrException;
> null:org.apache.solr.common.SolrException:
> org.apache.lucene.search.highlight.InvalidTokenOffsetsException: Token 0
> exceeds length of provided text sized 840
> at
> org.apache.solr.highlight.DefaultSolrHighlighter.doHighlightingByHighlighter(DefaultSolrHighlighter.java:542)
> at
> org.apache.solr.highlight.DefaultSolrHighlighter.doHighlighting(DefaultSolrHighlighter.java:414)
> at
> org.apache.solr.handler.component.HighlightComponent.process(HighlightComponent.java:139)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:208)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859)
> at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:703)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:406)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
> at org.eclipse.jetty.server.Server.handle(Server.java:368)
> at
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
> at
> org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
> at
> org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
> at
> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
> at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
> at
> org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
> at
> org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
> at java.lang.Thread.run(Unknown Source)
> Caused by: org.apache.lucene.search.highlight.InvalidTokenOffsetsException:
> Token 0 exceeds length of provided text sized 840
> at
> org.apache.lucene.search.highlight.Highlighter.getBestTextFragments(Highlighter.java:225)
> at
> org.apache.solr.highlight.DefaultSolrHighlighter.doHighlightingByHighlighter(DefaultSolrHighlighter.java:527)
> ... 33 more
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
