Re: Local Params handling (6.5->7.x)

2018-07-12 Thread Adam Constabaris
Thanks Joel, that's it!

I had tried all sorts of variants on the search terms but somehow missed
that issue (probably because my eyes glazed over before getting to the 7.2
upgrade notes).   While we were negatively impacted by the change, given
the existence of a simple workaround (anybody coming across the issue in
the future: defType=lucene in your query parameters), I don't think I'd
advocate for a behavioral change.  It might be worth adding a note to the
Local Params page in the reference guide, though, as I imagine edismax is a
fairly popular defType.

cheers,

AC

On Thu, Jul 12, 2018 at 12:37 PM Joel Bernstein  wrote:

> This may be the issue:
>
> https://issues.apache.org/jira/browse/SOLR-11501
>
> If it turns out that this causing the problem, please create a jira. It's
> important to discuss how SOLR-11501 is affecting real deployments.
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
> On Thu, Jul 12, 2018 at 12:30 PM, Adam Constabaris 
> wrote:
>
> > Hi folks,
> >
> > I am trying to track down what might have changed and where i could
> tweak a
> > configuration that worked under Solr 6.5.1 but which is not working as
> > expected under 7.3.1 or 7.4.0. We have default "qf" and "pf" defeined for
> > our select handler, along with "special" versions of those for use in
> > particular kinds of queries; our application (based on Project
> Blacklight)
> > make extensive use of local parameter expansion.
> >
> > Default query parser is edismax, and we are sending in queries like:
> >
> > "q={! qf=$special_qf pf=$special_pf}query terms'
> >
> > what used to happen, and what we expect given the documentation at
> >
> https://lucene.apache.org/solr/guide/6_6/local-parameters-in-queries.html
> > (6.6) and
> >
> https://lucene.apache.org/solr/guide/7_3/local-parameters-in-queries.html
> > (7.3) is that this should set qf and pf to their 'special' variants,
> where
> > the values are pulled from solrconfig.xml/confogoverlay.json (as
> > appropriate for the version).  We can achieve the indended effect via:
> >
> > q=query terms=${special_pf}=${special_qf}
> >
> > (and we have tested this and it seems to yield the expected behavior) but
> > this seems like it would be more involved than figuring out *why* the old
> > syntax isn't working and what we could change in our configset to make it
> > work again.  Our select handler configuration doesn't look particularly
> > weird:
> >
> > The actual values of qf/pf and special_qf/pf are really big, but I
> created
> > a 'smaller' configuration for the select handler:
> >
> >  "requestHandler": {  "/select": {"name": "/select",
> >  "class": "solr.SearchHandler","defaults": {
> > "defType": "edismax",  "echoParams": "explicit",
> > "rows": 10,  "q.alt": "*:*",  "mm": "6<90%",
> > "facet.mincount": "1",  "qf": "original_qf_notexpanded^1.5",
> >"pf": "original_pf_notexpanded^1.5",  "sample_qf":
> > "sample_qf_expanded^2",  "sample_pf": "sample_pf_expanded^2"
> >  }  },
> >
> >
> >
> > What we are seeing when we turn debugging on and look at the parsed query
> > is something like:
> >
> > "parsedquery":"+(DisjunctionMaxQuery(((original_qf_notexpanded:[[7b
> > 21] TO [7b 21]])^1.5))
> > DisjunctionMaxQuery(((original_qf_notexpanded:[[71 66 3d 24 73 61 6d
> > 70 6c 65 5f 71 66] TO [71 66 3d 24 73 61 6d 70 6c 65 5f 71 66]])^1.5))
> > DisjunctionMaxQuery(((original_qf_notexpanded:[[70 66 3d 24 73 61 6d
> > 70 6c 65 5f 70 66 7d 71 75 65 72 79] TO [70 66 3d 24 73 61 6d 70 6c 65
> > 5f 70 66 7d 71 75 65 72 79]])^1.5))
> > DisjunctionMaxQuery(((original_qf_notexpanded:[[74 65 72 6d 73] TO [74
> > 65 72 6d 73]])^1.5)))~4 ()",
> > "parsedquery_toString":"+original_qf_notexpanded:[[7b 21] TO
> > [7b 21]])^1.5) ((original_qf_notexpanded:[[71 66 3d 24 73 61 6d 70 6c
> > 65 5f 71 66] TO [71 66 3d 24 73 61 6d 70 6c 65 5f 71 66]])^1.5)
> > ((original_qf_notexpanded:[[70 66 3d 24 73 61 6d 70 6c 65 5f 70 66 7d
> > 71 75 65 72 79] TO [70 66 3d 24 73 61 6d 70 6c 65 5f 70 66 7d 71 75 65
> > 72 79]])^1.5) ((original_qf_notexpanded:[[74 65 72 6d 73] TO [74 65 72
> > 6d 73]])^1.5))~4) ()",
> >
> > the expanded fields are the ones from the default `qf` and `pf`
> > settings, and so i it looks like the local param syntax is not even
> > recognized, that it's somehow being "short-circuited".  Debug output
> > indicates edismax parser is still used in this case, and this is true
> > even if we change our query string to something like:
> >
> > q={!lucene ...}
> >
> > or
> >
> > q={!type=lucene ...}
> >
> > We've tried a number of variations, including
> >
> > (a) setting the 'sow' parameter to true and false, both inside the
> > expression and as a standalone parameter
> >
> > (b) changing the luceneMatchVersion in the solrconfig (originally it
> > was 6.0.0, have changed it to match Solr versions, both via
> > re-uploading teh configset or creating a copy and modifying it before
> > upload).
> >

Re: Local Params handling (6.5->7.x)

2018-07-12 Thread Joel Bernstein
This may be the issue:

https://issues.apache.org/jira/browse/SOLR-11501

If it turns out that this causing the problem, please create a jira. It's
important to discuss how SOLR-11501 is affecting real deployments.

Joel Bernstein
http://joelsolr.blogspot.com/

On Thu, Jul 12, 2018 at 12:30 PM, Adam Constabaris 
wrote:

> Hi folks,
>
> I am trying to track down what might have changed and where i could tweak a
> configuration that worked under Solr 6.5.1 but which is not working as
> expected under 7.3.1 or 7.4.0. We have default "qf" and "pf" defeined for
> our select handler, along with "special" versions of those for use in
> particular kinds of queries; our application (based on Project Blacklight)
> make extensive use of local parameter expansion.
>
> Default query parser is edismax, and we are sending in queries like:
>
> "q={! qf=$special_qf pf=$special_pf}query terms'
>
> what used to happen, and what we expect given the documentation at
> https://lucene.apache.org/solr/guide/6_6/local-parameters-in-queries.html
> (6.6) and
> https://lucene.apache.org/solr/guide/7_3/local-parameters-in-queries.html
> (7.3) is that this should set qf and pf to their 'special' variants, where
> the values are pulled from solrconfig.xml/confogoverlay.json (as
> appropriate for the version).  We can achieve the indended effect via:
>
> q=query terms=${special_pf}=${special_qf}
>
> (and we have tested this and it seems to yield the expected behavior) but
> this seems like it would be more involved than figuring out *why* the old
> syntax isn't working and what we could change in our configset to make it
> work again.  Our select handler configuration doesn't look particularly
> weird:
>
> The actual values of qf/pf and special_qf/pf are really big, but I created
> a 'smaller' configuration for the select handler:
>
>  "requestHandler": {  "/select": {"name": "/select",
>  "class": "solr.SearchHandler","defaults": {
> "defType": "edismax",  "echoParams": "explicit",
> "rows": 10,  "q.alt": "*:*",  "mm": "6<90%",
> "facet.mincount": "1",  "qf": "original_qf_notexpanded^1.5",
>"pf": "original_pf_notexpanded^1.5",  "sample_qf":
> "sample_qf_expanded^2",  "sample_pf": "sample_pf_expanded^2"
>  }  },
>
>
>
> What we are seeing when we turn debugging on and look at the parsed query
> is something like:
>
> "parsedquery":"+(DisjunctionMaxQuery(((original_qf_notexpanded:[[7b
> 21] TO [7b 21]])^1.5))
> DisjunctionMaxQuery(((original_qf_notexpanded:[[71 66 3d 24 73 61 6d
> 70 6c 65 5f 71 66] TO [71 66 3d 24 73 61 6d 70 6c 65 5f 71 66]])^1.5))
> DisjunctionMaxQuery(((original_qf_notexpanded:[[70 66 3d 24 73 61 6d
> 70 6c 65 5f 70 66 7d 71 75 65 72 79] TO [70 66 3d 24 73 61 6d 70 6c 65
> 5f 70 66 7d 71 75 65 72 79]])^1.5))
> DisjunctionMaxQuery(((original_qf_notexpanded:[[74 65 72 6d 73] TO [74
> 65 72 6d 73]])^1.5)))~4 ()",
> "parsedquery_toString":"+original_qf_notexpanded:[[7b 21] TO
> [7b 21]])^1.5) ((original_qf_notexpanded:[[71 66 3d 24 73 61 6d 70 6c
> 65 5f 71 66] TO [71 66 3d 24 73 61 6d 70 6c 65 5f 71 66]])^1.5)
> ((original_qf_notexpanded:[[70 66 3d 24 73 61 6d 70 6c 65 5f 70 66 7d
> 71 75 65 72 79] TO [70 66 3d 24 73 61 6d 70 6c 65 5f 70 66 7d 71 75 65
> 72 79]])^1.5) ((original_qf_notexpanded:[[74 65 72 6d 73] TO [74 65 72
> 6d 73]])^1.5))~4) ()",
>
> the expanded fields are the ones from the default `qf` and `pf`
> settings, and so i it looks like the local param syntax is not even
> recognized, that it's somehow being "short-circuited".  Debug output
> indicates edismax parser is still used in this case, and this is true
> even if we change our query string to something like:
>
> q={!lucene ...}
>
> or
>
> q={!type=lucene ...}
>
> We've tried a number of variations, including
>
> (a) setting the 'sow' parameter to true and false, both inside the
> expression and as a standalone parameter
>
> (b) changing the luceneMatchVersion in the solrconfig (originally it
> was 6.0.0, have changed it to match Solr versions, both via
> re-uploading teh configset or creating a copy and modifying it before
> upload).
>
> What else should I look at? Changing the searchcomponent stack?
>
> thanks!
>
> AC
>