Re: Unified highlighter returns an error when hl.fl param has undefined fields

[Yasufumi Mizoguchi]

Hi Shawn,

Thank you for your reply.

> that sounds like a bug in the argument parser that needs to be fixed.

I have created a JIRA about this.
https://issues.apache.org/jira/browse/SOLR-11334

Thanks,
Yasufumi



On 2017/09/06 9:48 PM, Shawn Heisey wrote:

On 9/4/2017 9:49 PM, Yasufumi Mizoguchi wrote:

I understood what you are saying. However, at least, I think it
strange that UnifiedSolrHighlighter
returns the same error when choosing ", " as the field delimiter in
hl.fl (e.g. hl.fl=name,%20manu).
This is because UnifiedSolrHighlighter detects that there is a
zero-length string between "," and " ",
and treats the string as a field name.
Is this a correct behavior?

I have never used any highlighter functionality, so I am clueless about
it ... but if the parsing of hl.fl leads to a situation where it thinks
an empty string is one of the field names that has been requested, that
sounds like a bug in the argument parser that needs to be fixed.

Thanks,
Shawn

Re: Unified highlighter returns an error when hl.fl param has undefined fields

[Shawn Heisey]

On 9/4/2017 9:49 PM, Yasufumi Mizoguchi wrote:
> I understood what you are saying. However, at least, I think it
> strange that UnifiedSolrHighlighter
> returns the same error when choosing ", " as the field delimiter in
> hl.fl (e.g. hl.fl=name,%20manu).
> This is because UnifiedSolrHighlighter detects that there is a
> zero-length string between "," and " ",
> and treats the string as a field name.
> Is this a correct behavior? 

I have never used any highlighter functionality, so I am clueless about
it ... but if the parsing of hl.fl leads to a situation where it thinks
an empty string is one of the field names that has been requested, that
sounds like a bug in the argument parser that needs to be fixed.

Thanks,
Shawn

Re: Unified highlighter returns an error when hl.fl param has undefined fields

[Yasufumi Mizoguchi]

Hi, Shawn,
(Sorry, I have sent this your private email address...)

Thanks for your reply.

I understood what you are saying. However, at least, I think it strange 
that UnifiedSolrHighlighter
returns the same error when choosing ", " as the field delimiter in 
hl.fl (e.g. hl.fl=name,%20manu).
This is because UnifiedSolrHighlighter detects that there is a 
zero-length string between "," and " ",

and treats the string as a field name.
Is this a correct behavior?

Thanks,
Yasufumi


On 2017/09/05 12:21 AM, Shawn Heisey wrote:

On 9/3/2017 10:31 PM, Yasufumi Mizoguchi wrote:

I am testing UnifiedHighlighter(hl.method=unified) with Solr 6.6 and
found that the highlighter returns following error when hl.fl
parameter has undefined fields.
The error occurs even if hl.fl parameter has ", "( + )
as a field delimiter. (e.g. hl.fl=name, manu)
Is this a bug? I think that the highlighter should set OffsetSource as
"NONE_NEEDED", if fields are not available

I think Solr is doing exactly what it should in this instance.  If you
ask Solr to use a field that it doesn't have, immediately returning an
error message is the right thing to do.  If Solr were to ignore the
error and proceed with the request anyway, then the person making the
request might never know that they have made a mistake.

Thanks,
Shawn

Re: Unified highlighter returns an error when hl.fl param has undefined fields

[Shawn Heisey]

On 9/3/2017 10:31 PM, Yasufumi Mizoguchi wrote:
> I am testing UnifiedHighlighter(hl.method=unified) with Solr 6.6 and
> found that the highlighter returns following error when hl.fl
> parameter has undefined fields.
> The error occurs even if hl.fl parameter has ", "( + )
> as a field delimiter. (e.g. hl.fl=name, manu)
> Is this a bug? I think that the highlighter should set OffsetSource as
> "NONE_NEEDED", if fields are not available 

I think Solr is doing exactly what it should in this instance.  If you
ask Solr to use a field that it doesn't have, immediately returning an
error message is the right thing to do.  If Solr were to ignore the
error and proceed with the request anyway, then the person making the
request might never know that they have made a mistake.

Thanks,
Shawn

Unified highlighter returns an error when hl.fl param has undefined fields

[Yasufumi Mizoguchi]

Hi,

I am testing UnifiedHighlighter(hl.method=unified) with Solr 6.6 and 
found that the highlighter returns following error when hl.fl parameter 
has undefined fields.
The error occurs even if hl.fl parameter has ", "( + ) as 
a field delimiter. (e.g. hl.fl=name, manu)
Is this a bug? I think that the highlighter should set OffsetSource as 
"NONE_NEEDED", if fields are not available


##Operations & Results

$ bin/solr -e techproducts
$ curl -XGET 
"http://localhost:8983/solr/techproducts/select?hl.fl=name,manu,undefined&hl.method=unified&hl=on&indent=on&q=corsair&wt=json&fl=name,manu";


{
  "responseHeader":{
    "status":400,
    "QTime":9,
    "params":{
  "q":"corsair",
  "hl":"on",
  "indent":"on",
  "fl":"name,manu,undefined",
  "hl.fl":"name,manu,undefined",
  "hl.method":"unified",
  "wt":"json"}},
  "response":{"numFound":2,"start":0,"docs":[
  {
    "name":"CORSAIR ValueSelect 1GB 184-Pin DDR SDRAM Unbuffered 
DDR 400 (PC 3200) System Memory - Retail",

    "manu":"Corsair Microsystems Inc."},
  {
    "name":"CORSAIR  XMS 2GB (2 x 1GB) 184-Pin DDR SDRAM Unbuffered 
DDR 400 (PC 3200) Dual Channel Kit System Memory - Retail",

    "manu":"Corsair Microsystems Inc."}]
  },
  "error":{
    "metadata":[
  "error-class","org.apache.solr.common.SolrException",
  "root-error-class","org.apache.solr.common.SolrException"],
    "msg":"undefined field undefined",
    "code":400}}


##Error messages

org.apache.solr.common.SolrException: undefined field undefined
at 
org.apache.solr.schema.IndexSchema.getDynamicFieldType(IndexSchema.java:1308)
at 
org.apache.solr.schema.IndexSchema$SolrIndexAnalyzer.getWrappedAnalyzer(IndexSchema.java:431)
at 
org.apache.lucene.analysis.AnalyzerWrapper.getOffsetGap(AnalyzerWrapper.java:149)
at 
org.apache.lucene.search.uhighlight.AnalysisOffsetStrategy.(AnalysisOffsetStrategy.java:43)
at 
org.apache.lucene.search.uhighlight.MemoryIndexOffsetStrategy.(MemoryIndexOffsetStrategy.java:56)
at 
org.apache.lucene.search.uhighlight.UnifiedHighlighter.getOffsetStrategy(UnifiedHighlighter.java:841)
at 
org.apache.lucene.search.uhighlight.UnifiedHighlighter.getFieldHighlighter(UnifiedHighlighter.java:750)
at 
org.apache.lucene.search.uhighlight.UnifiedHighlighter.highlightFieldsAsObjects(UnifiedHighlighter.java:573)
at 
org.apache.lucene.search.uhighlight.UnifiedHighlighter.highlightFields(UnifiedHighlighter.java:507)
at 
org.apache.solr.highlight.UnifiedSolrHighlighter.doHighlighting(UnifiedSolrHighlighter.java:146)
at 
org.apache.solr.handler.component.HighlightComponent.process(HighlightComponent.java:182)
at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2440)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:723)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:529)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:298)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)
at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at 
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at 
org.eclips