Re: Boosting on field empty or not
The problem with what you provided is it is boosting ALL documents whether the field is empty or not On Tue, May 15, 2012 at 3:52 AM, Ahmet Arslan iori...@yahoo.com wrote: Basically I want documents that have a given field populated to have a higher score than the documents that dont. So if you search for foo I want documents that contain foo, but i want the documents that have field a populated to have a higher score... Hi Donald, Since you are using edismax, it is better to use bq (boosting query) for this. bq=reqularprice:[* TO *]^50 http://wiki.apache.org/solr/DisMaxQParserPlugin#bq_.28Boost_Query.29 defType=edismaxqf=nameSuggest^10 name^10 codeTXT^2 description^1 brand_search^0 cat_search^10q=chairsbq=reqularprice:[* TO *]^50
Re: Boosting on field empty or not
I have figured it out using your recommendation...I just had to give it a high enough boost. BTW its a float field On Tue, May 15, 2012 at 9:21 AM, Ahmet Arslan iori...@yahoo.com wrote: The problem with what you provided is it is boosting ALL documents whether the field is empty or not Then all of your fields are non-empty? What is the type of your field?
Re: Boosting on field empty or not
Scratch that...it still seems to be boosting documents where the value of the field is empty. bq=regularprice:[0.01 TO *]^50 Results with bq set: doc float name=score2.2172112/float str name=codebhl-ltab-30/str /doc Results without bq set: doc float name=score2.4847748/float str name=codebhl-ltab-30/str /doc On Tue, May 15, 2012 at 12:40 PM, Donald Organ dor...@donaldorgan.comwrote: I have figured it out using your recommendation...I just had to give it a high enough boost. BTW its a float field On Tue, May 15, 2012 at 9:21 AM, Ahmet Arslan iori...@yahoo.com wrote: The problem with what you provided is it is boosting ALL documents whether the field is empty or not Then all of your fields are non-empty? What is the type of your field?
Re: Boosting on field empty or not
If the bq is only supposed apply the boost when the field value is greater than 0.01 why would trying another query make sure this is working. Its applying the boost to all the fields, yes when the boost is high enough most of documents with a value GT 0.01 show up first however since it is applying the boost to all the documents sometimes documents without a value in this field appear before those that do. On Tue, May 15, 2012 at 4:51 PM, Ahmet Arslan iori...@yahoo.com wrote: Scratch that...it still seems to be boosting documents where the value of the field is empty. bq=regularprice:[0.01 TO *]^50 Results with bq set: doc float name=score2.2172112/float str name=codebhl-ltab-30/str /doc Results without bq set: doc float name=score2.4847748/float str name=codebhl-ltab-30/str /doc Important thing is the order. Does the order of results change in a way that you want? (When you add bq) It is not a good idea to compare scores of two different queries. I *think* queryNorm is causing this difference. You can add debugQuery=on and see what is the difference.
Boosting on field empty or not
Is there a way to boost a document based on whether the field is empty or not. I am looking to boost documents that have a specific field populated.
Re: Boosting on field empty or not
OK maybe i need to describe this a little more. Basically I want documents that have a given field populated to have a higher score than the documents that dont. So if you search for foo I want documents that contain foo, but i want the documents that have field a populated to have a higher score... Is there a way to do this? On Mon, May 14, 2012 at 2:22 PM, Jack Krupansky j...@basetechnology.comwrote: In a query or filter query you can write +field:* to require that a field be populated or +(-field:*) to require that it not be populated -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 2:10 PM To: solr-user Subject: Boosting on field empty or not Is there a way to boost a document based on whether the field is empty or not. I am looking to boost documents that have a specific field populated.
Re: Boosting on field empty or not
OK i just tried: q=chairs AND (regularprice:*^5 OR (-regularprice:*)5) And that gives me 0 results On Mon, May 14, 2012 at 2:51 PM, Jack Krupansky j...@basetechnology.comwrote: foo AND (field:*^2.0 OR (-field:*)^0.5) So, if a doc has anything in the field, it gets boosted, and if the doc does not have anything in the field, de-boost it. Choose the boost factors to suit your desired boosting effect. -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 2:38 PM To: solr-user@lucene.apache.org Subject: Re: Boosting on field empty or not OK maybe i need to describe this a little more. Basically I want documents that have a given field populated to have a higher score than the documents that dont. So if you search for foo I want documents that contain foo, but i want the documents that have field a populated to have a higher score... Is there a way to do this? On Mon, May 14, 2012 at 2:22 PM, Jack Krupansky j...@basetechnology.com* *wrote: In a query or filter query you can write +field:* to require that a field be populated or +(-field:*) to require that it not be populated -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 2:10 PM To: solr-user Subject: Boosting on field empty or not Is there a way to boost a document based on whether the field is empty or not. I am looking to boost documents that have a specific field populated.
Re: Boosting on field empty or not
Still doesnt appear to be working. Here is the full Query string: defType=edismaxstart=0rows=24facet=trueqf=nameSuggest^10 name^10 codeTXT^2 description^1 brand_search^0 cat_search^10spellcheck=truespellcheck.collate=truespellcheck.q=chairsfacet.mincount=1fl=code,scoreq=chairs AND (regularprice:*^5 OR (*:* -regularprice:*)5) On Mon, May 14, 2012 at 3:28 PM, Jack Krupansky j...@basetechnology.comwrote: Sorry, make that: q=chairs AND (regularprice:*^5 OR (*:* -regularprice:*)^0.5) I forgot that pure negative queries are broken again, so you need the *:* in there. I noticed that you second boost operator was missing as well. -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 3:24 PM To: solr-user@lucene.apache.org Subject: Re: Boosting on field empty or not OK i just tried: q=chairs AND (regularprice:*^5 OR (-regularprice:*)5) And that gives me 0 results On Mon, May 14, 2012 at 2:51 PM, Jack Krupansky j...@basetechnology.com* *wrote: foo AND (field:*^2.0 OR (-field:*)^0.5) So, if a doc has anything in the field, it gets boosted, and if the doc does not have anything in the field, de-boost it. Choose the boost factors to suit your desired boosting effect. -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 2:38 PM To: solr-user@lucene.apache.org Subject: Re: Boosting on field empty or not OK maybe i need to describe this a little more. Basically I want documents that have a given field populated to have a higher score than the documents that dont. So if you search for foo I want documents that contain foo, but i want the documents that have field a populated to have a higher score... Is there a way to do this? On Mon, May 14, 2012 at 2:22 PM, Jack Krupansky j...@basetechnology.com * *wrote: In a query or filter query you can write +field:* to require that a field be populated or +(-field:*) to require that it not be populated -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 2:10 PM To: solr-user Subject: Boosting on field empty or not Is there a way to boost a document based on whether the field is empty or not. I am looking to boost documents that have a specific field populated.
Re: Boosting on field empty or not
OK i must be missing something: defType=edismaxstart=0rows=24facet=trueqf=nameSuggest^10 name^10 codeTXT^2 description^1 brand_search^0 cat_search^10spellcheck=truespellcheck.collate=truespellcheck.q=chairsfacet.mincount=1fl=code,scoreq=chairs AND (regularprice:*^5 OR (*:* -regularprice:*)^5)sort=score desc On Mon, May 14, 2012 at 3:36 PM, Jack Krupansky j...@basetechnology.comwrote: (*:* -regularprice:*)5 should be (*:* -regularprice:*)^0.5 - the missing boost operator. -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 3:31 PM To: solr-user@lucene.apache.org Subject: Re: Boosting on field empty or not Still doesnt appear to be working. Here is the full Query string: defType=edismaxstart=0rows=**24facet=trueqf=nameSuggest^**10 name^10 codeTXT^2 description^1 brand_search^0 cat_search^10spellcheck=true**spellcheck.collate=true** spellcheck.q=chairsfacet.**mincount=1fl=code,scoreq=**chairs AND (regularprice:*^5 OR (*:* -regularprice:*)5) On Mon, May 14, 2012 at 3:28 PM, Jack Krupansky j...@basetechnology.com* *wrote: Sorry, make that: q=chairs AND (regularprice:*^5 OR (*:* -regularprice:*)^0.5) I forgot that pure negative queries are broken again, so you need the *:* in there. I noticed that you second boost operator was missing as well. -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 3:24 PM To: solr-user@lucene.apache.org Subject: Re: Boosting on field empty or not OK i just tried: q=chairs AND (regularprice:*^5 OR (-regularprice:*)5) And that gives me 0 results On Mon, May 14, 2012 at 2:51 PM, Jack Krupansky j...@basetechnology.com * *wrote: foo AND (field:*^2.0 OR (-field:*)^0.5) So, if a doc has anything in the field, it gets boosted, and if the doc does not have anything in the field, de-boost it. Choose the boost factors to suit your desired boosting effect. -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 2:38 PM To: solr-user@lucene.apache.org Subject: Re: Boosting on field empty or not OK maybe i need to describe this a little more. Basically I want documents that have a given field populated to have a higher score than the documents that dont. So if you search for foo I want documents that contain foo, but i want the documents that have field a populated to have a higher score... Is there a way to do this? On Mon, May 14, 2012 at 2:22 PM, Jack Krupansky j...@basetechnology.com * *wrote: In a query or filter query you can write +field:* to require that a field be populated or +(-field:*) to require that it not be populated -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 2:10 PM To: solr-user Subject: Boosting on field empty or not Is there a way to boost a document based on whether the field is empty or not. I am looking to boost documents that have a specific field populated.
Re: Boosting on field empty or not
OK it looks like the query change is working but it looks like it boosting everything even documents that have that field empty On Mon, May 14, 2012 at 3:41 PM, Donald Organ dor...@donaldorgan.comwrote: OK i must be missing something: defType=edismaxstart=0rows=24facet=trueqf=nameSuggest^10 name^10 codeTXT^2 description^1 brand_search^0 cat_search^10spellcheck=truespellcheck.collate=truespellcheck.q=chairsfacet.mincount=1fl=code,scoreq=chairs AND (regularprice:*^5 OR (*:* -regularprice:*)^5)sort=score desc On Mon, May 14, 2012 at 3:36 PM, Jack Krupansky j...@basetechnology.comwrote: (*:* -regularprice:*)5 should be (*:* -regularprice:*)^0.5 - the missing boost operator. -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 3:31 PM To: solr-user@lucene.apache.org Subject: Re: Boosting on field empty or not Still doesnt appear to be working. Here is the full Query string: defType=edismaxstart=0rows=**24facet=trueqf=nameSuggest^**10 name^10 codeTXT^2 description^1 brand_search^0 cat_search^10spellcheck=true**spellcheck.collate=true** spellcheck.q=chairsfacet.**mincount=1fl=code,scoreq=**chairs AND (regularprice:*^5 OR (*:* -regularprice:*)5) On Mon, May 14, 2012 at 3:28 PM, Jack Krupansky j...@basetechnology.com **wrote: Sorry, make that: q=chairs AND (regularprice:*^5 OR (*:* -regularprice:*)^0.5) I forgot that pure negative queries are broken again, so you need the *:* in there. I noticed that you second boost operator was missing as well. -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 3:24 PM To: solr-user@lucene.apache.org Subject: Re: Boosting on field empty or not OK i just tried: q=chairs AND (regularprice:*^5 OR (-regularprice:*)5) And that gives me 0 results On Mon, May 14, 2012 at 2:51 PM, Jack Krupansky j...@basetechnology.com * *wrote: foo AND (field:*^2.0 OR (-field:*)^0.5) So, if a doc has anything in the field, it gets boosted, and if the doc does not have anything in the field, de-boost it. Choose the boost factors to suit your desired boosting effect. -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 2:38 PM To: solr-user@lucene.apache.org Subject: Re: Boosting on field empty or not OK maybe i need to describe this a little more. Basically I want documents that have a given field populated to have a higher score than the documents that dont. So if you search for foo I want documents that contain foo, but i want the documents that have field a populated to have a higher score... Is there a way to do this? On Mon, May 14, 2012 at 2:22 PM, Jack Krupansky j...@basetechnology.com * *wrote: In a query or filter query you can write +field:* to require that a field be populated or +(-field:*) to require that it not be populated -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 2:10 PM To: solr-user Subject: Boosting on field empty or not Is there a way to boost a document based on whether the field is empty or not. I am looking to boost documents that have a specific field populated.
Re: Boosting on field empty or not
q=chairs AND (regularprice:*^5 OR (*:* -regularprice:*)^0.5)sort=score desc Same effect. On Mon, May 14, 2012 at 4:12 PM, Jack Krupansky j...@basetechnology.comwrote: Change the second boost to 0.5 to de-boost doc that are missing the field value. You had them the same. -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 4:01 PM To: solr-user@lucene.apache.org Subject: Re: Boosting on field empty or not OK it looks like the query change is working but it looks like it boosting everything even documents that have that field empty On Mon, May 14, 2012 at 3:41 PM, Donald Organ dor...@donaldorgan.com wrote: OK i must be missing something: defType=edismaxstart=0rows=**24facet=trueqf=nameSuggest^**10 name^10 codeTXT^2 description^1 brand_search^0 cat_search^10spellcheck=true** spellcheck.collate=true**spellcheck.q=chairsfacet.** mincount=1fl=code,scoreq=**chairs AND (regularprice:*^5 OR (*:* -regularprice:*)^5)sort=score desc On Mon, May 14, 2012 at 3:36 PM, Jack Krupansky j...@basetechnology.com **wrote: (*:* -regularprice:*)5 should be (*:* -regularprice:*)^0.5 - the missing boost operator. -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 3:31 PM To: solr-user@lucene.apache.org Subject: Re: Boosting on field empty or not Still doesnt appear to be working. Here is the full Query string: defType=edismaxstart=0rows=24facet=trueqf=nameSuggest^10 name^10 codeTXT^2 description^1 brand_search^0 cat_search^10spellcheck=truespellcheck.collate=true** spellcheck.q=chairsfacet.mincount=1fl=code,scoreq=chairs AND (regularprice:*^5 OR (*:* -regularprice:*)5) On Mon, May 14, 2012 at 3:28 PM, Jack Krupansky j...@basetechnology.com **wrote: Sorry, make that: q=chairs AND (regularprice:*^5 OR (*:* -regularprice:*)^0.5) I forgot that pure negative queries are broken again, so you need the *:* in there. I noticed that you second boost operator was missing as well. -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 3:24 PM To: solr-user@lucene.apache.org Subject: Re: Boosting on field empty or not OK i just tried: q=chairs AND (regularprice:*^5 OR (-regularprice:*)5) And that gives me 0 results On Mon, May 14, 2012 at 2:51 PM, Jack Krupansky j...@basetechnology.com * *wrote: foo AND (field:*^2.0 OR (-field:*)^0.5) So, if a doc has anything in the field, it gets boosted, and if the doc does not have anything in the field, de-boost it. Choose the boost factors to suit your desired boosting effect. -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 2:38 PM To: solr-user@lucene.apache.org Subject: Re: Boosting on field empty or not OK maybe i need to describe this a little more. Basically I want documents that have a given field populated to have a higher score than the documents that dont. So if you search for foo I want documents that contain foo, but i want the documents that have field a populated to have a higher score... Is there a way to do this? On Mon, May 14, 2012 at 2:22 PM, Jack Krupansky j...@basetechnology.com * *wrote: In a query or filter query you can write +field:* to require that a field be populated or +(-field:*) to require that it not be populated -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 2:10 PM To: solr-user Subject: Boosting on field empty or not Is there a way to boost a document based on whether the field is empty or not. I am looking to boost documents that have a specific field populated.
Re: Boosting on field empty or not
I've even tried upping the boost to 10 and the de-boost to 1but yet its still applying the boost to all the documents returned. So it matter if this is a money field? On Mon, May 14, 2012 at 4:19 PM, Donald Organ dor...@donaldorgan.comwrote: q=chairs AND (regularprice:*^5 OR (*:* -regularprice:*)^0.5)sort=score desc Same effect. On Mon, May 14, 2012 at 4:12 PM, Jack Krupansky j...@basetechnology.comwrote: Change the second boost to 0.5 to de-boost doc that are missing the field value. You had them the same. -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 4:01 PM To: solr-user@lucene.apache.org Subject: Re: Boosting on field empty or not OK it looks like the query change is working but it looks like it boosting everything even documents that have that field empty On Mon, May 14, 2012 at 3:41 PM, Donald Organ dor...@donaldorgan.com wrote: OK i must be missing something: defType=edismaxstart=0rows=**24facet=trueqf=nameSuggest^**10 name^10 codeTXT^2 description^1 brand_search^0 cat_search^10spellcheck=true**spellcheck.collate=true** spellcheck.q=chairsfacet.**mincount=1fl=code,scoreq=**chairs AND (regularprice:*^5 OR (*:* -regularprice:*)^5)sort=score desc On Mon, May 14, 2012 at 3:36 PM, Jack Krupansky j...@basetechnology.com **wrote: (*:* -regularprice:*)5 should be (*:* -regularprice:*)^0.5 - the missing boost operator. -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 3:31 PM To: solr-user@lucene.apache.org Subject: Re: Boosting on field empty or not Still doesnt appear to be working. Here is the full Query string: defType=edismaxstart=0rows=24facet=trueqf=nameSuggest^10 name^10 codeTXT^2 description^1 brand_search^0 cat_search^10spellcheck=truespellcheck.collate=true** spellcheck.q=chairsfacet.mincount=1fl=code,scoreq=chairs AND (regularprice:*^5 OR (*:* -regularprice:*)5) On Mon, May 14, 2012 at 3:28 PM, Jack Krupansky j...@basetechnology.com **wrote: Sorry, make that: q=chairs AND (regularprice:*^5 OR (*:* -regularprice:*)^0.5) I forgot that pure negative queries are broken again, so you need the *:* in there. I noticed that you second boost operator was missing as well. -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 3:24 PM To: solr-user@lucene.apache.org Subject: Re: Boosting on field empty or not OK i just tried: q=chairs AND (regularprice:*^5 OR (-regularprice:*)5) And that gives me 0 results On Mon, May 14, 2012 at 2:51 PM, Jack Krupansky j...@basetechnology.com * *wrote: foo AND (field:*^2.0 OR (-field:*)^0.5) So, if a doc has anything in the field, it gets boosted, and if the doc does not have anything in the field, de-boost it. Choose the boost factors to suit your desired boosting effect. -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 2:38 PM To: solr-user@lucene.apache.org Subject: Re: Boosting on field empty or not OK maybe i need to describe this a little more. Basically I want documents that have a given field populated to have a higher score than the documents that dont. So if you search for foo I want documents that contain foo, but i want the documents that have field a populated to have a higher score... Is there a way to do this? On Mon, May 14, 2012 at 2:22 PM, Jack Krupansky j...@basetechnology.com * *wrote: In a query or filter query you can write +field:* to require that a field be populated or +(-field:*) to require that it not be populated -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 2:10 PM To: solr-user Subject: Boosting on field empty or not Is there a way to boost a document based on whether the field is empty or not. I am looking to boost documents that have a specific field populated.
Re: Boosting on field empty or not
OK i think i found the proper way to do what i was trying to do: q=chairs AND (regularprice:[0 TO *]^5) On Mon, May 14, 2012 at 4:25 PM, Donald Organ dor...@donaldorgan.comwrote: I've even tried upping the boost to 10 and the de-boost to 1but yet its still applying the boost to all the documents returned. So it matter if this is a money field? On Mon, May 14, 2012 at 4:19 PM, Donald Organ dor...@donaldorgan.comwrote: q=chairs AND (regularprice:*^5 OR (*:* -regularprice:*)^0.5)sort=score desc Same effect. On Mon, May 14, 2012 at 4:12 PM, Jack Krupansky j...@basetechnology.comwrote: Change the second boost to 0.5 to de-boost doc that are missing the field value. You had them the same. -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 4:01 PM To: solr-user@lucene.apache.org Subject: Re: Boosting on field empty or not OK it looks like the query change is working but it looks like it boosting everything even documents that have that field empty On Mon, May 14, 2012 at 3:41 PM, Donald Organ dor...@donaldorgan.com wrote: OK i must be missing something: defType=edismaxstart=0rows=**24facet=trueqf=nameSuggest^**10 name^10 codeTXT^2 description^1 brand_search^0 cat_search^10spellcheck=true**spellcheck.collate=true** spellcheck.q=chairsfacet.**mincount=1fl=code,scoreq=**chairs AND (regularprice:*^5 OR (*:* -regularprice:*)^5)sort=score desc On Mon, May 14, 2012 at 3:36 PM, Jack Krupansky j...@basetechnology.com**wrote: (*:* -regularprice:*)5 should be (*:* -regularprice:*)^0.5 - the missing boost operator. -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 3:31 PM To: solr-user@lucene.apache.org Subject: Re: Boosting on field empty or not Still doesnt appear to be working. Here is the full Query string: defType=edismaxstart=0rows=24facet=trueqf=nameSuggest^10 name^10 codeTXT^2 description^1 brand_search^0 cat_search^10spellcheck=truespellcheck.collate=true** spellcheck.q=chairsfacet.mincount=1fl=code,scoreq=chairs AND (regularprice:*^5 OR (*:* -regularprice:*)5) On Mon, May 14, 2012 at 3:28 PM, Jack Krupansky j...@basetechnology.com **wrote: Sorry, make that: q=chairs AND (regularprice:*^5 OR (*:* -regularprice:*)^0.5) I forgot that pure negative queries are broken again, so you need the *:* in there. I noticed that you second boost operator was missing as well. -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 3:24 PM To: solr-user@lucene.apache.org Subject: Re: Boosting on field empty or not OK i just tried: q=chairs AND (regularprice:*^5 OR (-regularprice:*)5) And that gives me 0 results On Mon, May 14, 2012 at 2:51 PM, Jack Krupansky j...@basetechnology.com * *wrote: foo AND (field:*^2.0 OR (-field:*)^0.5) So, if a doc has anything in the field, it gets boosted, and if the doc does not have anything in the field, de-boost it. Choose the boost factors to suit your desired boosting effect. -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 2:38 PM To: solr-user@lucene.apache.org Subject: Re: Boosting on field empty or not OK maybe i need to describe this a little more. Basically I want documents that have a given field populated to have a higher score than the documents that dont. So if you search for foo I want documents that contain foo, but i want the documents that have field a populated to have a higher score... Is there a way to do this? On Mon, May 14, 2012 at 2:22 PM, Jack Krupansky j...@basetechnology.com * *wrote: In a query or filter query you can write +field:* to require that a field be populated or +(-field:*) to require that it not be populated -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 2:10 PM To: solr-user Subject: Boosting on field empty or not Is there a way to boost a document based on whether the field is empty or not. I am looking to boost documents that have a specific field populated.
Re: Boosting on field empty or not
OK thats giving me only documents that have the field populated On Mon, May 14, 2012 at 4:28 PM, Donald Organ dor...@donaldorgan.comwrote: OK i think i found the proper way to do what i was trying to do: q=chairs AND (regularprice:[0 TO *]^5) On Mon, May 14, 2012 at 4:25 PM, Donald Organ dor...@donaldorgan.comwrote: I've even tried upping the boost to 10 and the de-boost to 1but yet its still applying the boost to all the documents returned. So it matter if this is a money field? On Mon, May 14, 2012 at 4:19 PM, Donald Organ dor...@donaldorgan.comwrote: q=chairs AND (regularprice:*^5 OR (*:* -regularprice:*)^0.5)sort=score desc Same effect. On Mon, May 14, 2012 at 4:12 PM, Jack Krupansky j...@basetechnology.com wrote: Change the second boost to 0.5 to de-boost doc that are missing the field value. You had them the same. -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 4:01 PM To: solr-user@lucene.apache.org Subject: Re: Boosting on field empty or not OK it looks like the query change is working but it looks like it boosting everything even documents that have that field empty On Mon, May 14, 2012 at 3:41 PM, Donald Organ dor...@donaldorgan.com wrote: OK i must be missing something: defType=edismaxstart=0rows=**24facet=trueqf=nameSuggest^**10 name^10 codeTXT^2 description^1 brand_search^0 cat_search^10spellcheck=true**spellcheck.collate=true** spellcheck.q=chairsfacet.**mincount=1fl=code,scoreq=**chairs AND (regularprice:*^5 OR (*:* -regularprice:*)^5)sort=score desc On Mon, May 14, 2012 at 3:36 PM, Jack Krupansky j...@basetechnology.com**wrote: (*:* -regularprice:*)5 should be (*:* -regularprice:*)^0.5 - the missing boost operator. -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 3:31 PM To: solr-user@lucene.apache.org Subject: Re: Boosting on field empty or not Still doesnt appear to be working. Here is the full Query string: defType=edismaxstart=0rows=24facet=trueqf=nameSuggest^10 name^10 codeTXT^2 description^1 brand_search^0 cat_search^10spellcheck=truespellcheck.collate=true** spellcheck.q=chairsfacet.mincount=1fl=code,scoreq=chairs AND (regularprice:*^5 OR (*:* -regularprice:*)5) On Mon, May 14, 2012 at 3:28 PM, Jack Krupansky j...@basetechnology.com **wrote: Sorry, make that: q=chairs AND (regularprice:*^5 OR (*:* -regularprice:*)^0.5) I forgot that pure negative queries are broken again, so you need the *:* in there. I noticed that you second boost operator was missing as well. -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 3:24 PM To: solr-user@lucene.apache.org Subject: Re: Boosting on field empty or not OK i just tried: q=chairs AND (regularprice:*^5 OR (-regularprice:*)5) And that gives me 0 results On Mon, May 14, 2012 at 2:51 PM, Jack Krupansky j...@basetechnology.com * *wrote: foo AND (field:*^2.0 OR (-field:*)^0.5) So, if a doc has anything in the field, it gets boosted, and if the doc does not have anything in the field, de-boost it. Choose the boost factors to suit your desired boosting effect. -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 2:38 PM To: solr-user@lucene.apache.org Subject: Re: Boosting on field empty or not OK maybe i need to describe this a little more. Basically I want documents that have a given field populated to have a higher score than the documents that dont. So if you search for foo I want documents that contain foo, but i want the documents that have field a populated to have a higher score... Is there a way to do this? On Mon, May 14, 2012 at 2:22 PM, Jack Krupansky j...@basetechnology.com * *wrote: In a query or filter query you can write +field:* to require that a field be populated or +(-field:*) to require that it not be populated -- Jack Krupansky -Original Message- From: Donald Organ Sent: Monday, May 14, 2012 2:10 PM To: solr-user Subject: Boosting on field empty or not Is there a way to boost a document based on whether the field is empty or not. I am looking to boost documents that have a specific field populated.
Boosting documents based on search term/phrase
Is there a way to boost documents based on the search term/phrase?
Re: Boosting documents based on search term/phrase
query elevation was exactly what I was talking about. Now is there a way to add this to the default query handler? On Tue, May 1, 2012 at 4:26 PM, Jack Krupansky j...@basetechnology.comwrote: Do you mean besides query elevation? http://wiki.apache.org/solr/**QueryElevationComponenthttp://wiki.apache.org/solr/QueryElevationComponent And besides explicit boosting by the user (the ^ suffix operator after a term/phrase)? -- Jack Krupansky -Original Message- From: Donald Organ Sent: Tuesday, May 01, 2012 3:59 PM To: solr-user Subject: Boosting documents based on search term/phrase Is there a way to boost documents based on the search term/phrase?
Re: Boosting documents based on search term/phrase
Perfect, this is working well. On Tue, May 1, 2012 at 5:33 PM, Jeevanandam je...@myjeeva.com wrote: Yes, you can add in last-components section on default query handler. arr name=last-components strelevator/str /arr - Jeevanandam On 02-05-2012 3:53 am, Donald Organ wrote: query elevation was exactly what I was talking about. Now is there a way to add this to the default query handler? On Tue, May 1, 2012 at 4:26 PM, Jack Krupansky j...@basetechnology.com**wrote: Do you mean besides query elevation? http://wiki.apache.org/solr/QueryElevationComponenthttp://wiki.apache.org/solr/**QueryElevationComponent http:/**/wiki.apache.org/solr/**QueryElevationComponenthttp://wiki.apache.org/solr/QueryElevationComponent And besides explicit boosting by the user (the ^ suffix operator after a term/phrase)? -- Jack Krupansky -Original Message- From: Donald Organ Sent: Tuesday, May 01, 2012 3:59 PM To: solr-user Subject: Boosting documents based on search term/phrase Is there a way to boost documents based on the search term/phrase?
Reporting tools
Are there any reporting tools out there? So I can analyzer search term frequency, filter frequency, etc?
Need some quick help diagnosing query
Right now i am doing the following: qf=name^1.75 codeTXT^1.75 cat_search^1.5 description^0.8 brand^5.0 cat_search^0.8 fl=code,score defType=dismax q=whitney brothers carts if i change it to the following then i get results: qf=name^1.75 codeTXT^1.75 cat_search^1.5 description^0.8 brand^5.0 cat_search^0.8 fl=code,score defType=dismax q=whitney brothers So why is the first query returning 0 results?
Re: Need some quick help diagnosing query
Simply your collection does contain a doc having all these three terms? Try different mm values. http://wiki.apache.org/solr/DisMaxQParserPlugin#mm_.28Minimum_.27Should.27_Match.29 Would this also be affected if one of the fields that contains that term is a defined as solr.StrField where as most of the other fields are defined as solr.TextField?
Re: Need some quick help diagnosing query
Would this also be affected if one of the fields that contains that term is a defined as solr.StrField where as most of the other fields are defined as solr.TextField? It could be. string fields are not analyzed. For example, one whitespace can prevent match. Cards and cards wont match too. (lowercase) Ok looks like its time to setup some copyFields. I will try that and see if that help and point my query at the solr.TextField's
Re: Help with Synonyms
Hi Donald, Try to remove tokenizerFactory=**KeywordTokenizerFactory in your synonym filter definition because I think you would want to tokenize the synonym settings in synonyms.txt as floor / locker = storage / locker. But if you set it to KeywordTokenizer, it will be a map of floor locker = storage locker, and as you are using WhitespaceTokenizer for your tokenizer/ in analyzer/, then if you try to index floor locker, it will be floor/locker (not floor locker), as a result, it will not match to your synonym map. Aside, I recommend that you would set charFilter/ - tokenizer/ - filter/ chain in the natural order in analyzer/, though if those are wrong it won't be the cause of the problem at all. OK so I have updated my schema.xml to the following: fieldType name=text class=solr.TextField positionIncrementGap=100 omitNorms=false analyzer type=index charFilter class=solr.HTMLStripCharFilterFactory/ tokenizer class=solr.WhitespaceTokenizerFactory/ filter class=solr.SynonymFilterFactory synonyms=synonyms.txt ignoreCase=true expand=false/ filter class=solr.WordDelimiterFilterFactory generateWordParts=1 generateNumberParts=1 catenateWords=1 catenateNumbers=1 catenateAll=0 / filter class=solr.StopFilterFactory ignoreCase=true words=stopwords.txt / filter class=solr.EnglishPorterFilterFactory protected=protwords.txt / filter class=solr.LowerCaseFilterFactory / filter class=solr.RemoveDuplicatesTokenFilterFactory / /analyzer . I am still getting results for storage locker and no results for floor locker synonyms.txt still looks like this: floor locker=storage locker
Re: Help with Synonyms
No I do synonyms at index time. On Monday, March 5, 2012, Koji Sekiguchi k...@r.email.ne.jp wrote: (12/03/06 0:11), Donald Organ wrote: Try to remove tokenizerFactory=**KeywordTokenizerFactory in your synonym filter definition because I think you would want to tokenize the synonym settings in synonyms.txt as floor / locker = storage / locker. But if you set it to KeywordTokenizer, it will be a map of floor locker = storage locker, and as you are using WhitespaceTokenizer for yourtokenizer/ inanalyzer/, then if you try to index floor locker, it will be floor/locker (not floor locker), as a result, it will not match to your synonym map. Aside, I recommend that you would setcharFilter/ -tokenizer/ - filter/ chain in the natural order inanalyzer/, though if those are wrong it won't be the cause of the problem at all. OK so I have updated my schema.xml to the following: fieldType name=text class=solr.TextField positionIncrementGap=100 omitNorms=false analyzer type=index charFilter class=solr.HTMLStripCharFilterFactory/ tokenizer class=solr.WhitespaceTokenizerFactory/ filter class=solr.SynonymFilterFactory synonyms=synonyms.txt ignoreCase=true expand=false/ filter class=solr.WordDelimiterFilterFactory generateWordParts=1 generateNumberParts=1 catenateWords=1 catenateNumbers=1 catenateAll=0 / filter class=solr.StopFilterFactory ignoreCase=true words=stopwords.txt / filter class=solr.EnglishPorterFilterFactory protected=protwords.txt / filter class=solr.LowerCaseFilterFactory / filter class=solr.RemoveDuplicatesTokenFilterFactory / /analyzer . I am still getting results for storage locker and no results for floor locker synonyms.txt still looks like this: floor locker=storage locker Hi Donald, Do you use same SynonymFilter setting to the query analyzer part (analyzer type=query)? koji -- Query Log Visualizer for Apache Solr http://soleami.com/
Re: Help with Synonyms
Ok so do I need to use a different format in my synonyms.txt file in order to do this at index time? On Monday, March 5, 2012, Koji Sekiguchi k...@r.email.ne.jp wrote: (12/03/06 11:07), Donald Organ wrote: No I do synonyms at index time. : I am still getting results for storage locker and no results for floor locker synonyms.txt still looks like this: floor locker=storage locker So that's the cause of the problem. Due to the definition floor locker=storage locker on index time analysis, you got storage / locker in your index, no floor terms in your index at all. In general, if you use = method in your synonyms.txt, you should apply same rule to both index and query time. koji -- Query Log Visualizer for Apache Solr http://soleami.com/
Re: Help with Synonyms
Excellent thank you, it is now working! On Mon, Mar 5, 2012 at 9:37 PM, Koji Sekiguchi k...@r.email.ne.jp wrote: (12/03/06 11:23), Donald Organ wrote: Ok so do I need to use a different format in my synonyms.txt file in order to do this at index time? Right, if you want to apply synonym rules to only index time. Use , like this: floor locker, storage locker And don't forget to set expand=true in your index time synonym definition. This makes if you have floor locker in your document, it will be expanded not only floor locker but also storage locker in index, then you can search the document by any of q=floor locker or storage locker. koji -- Query Log Visualizer for Apache Solr http://soleami.com/
Help with Synonyms
I am trying to get synonyms working correctly, I want to map floor locker tostorage locker currently searching for storage locker produces results were as searching for floor locker does not produce any results. I have the following setup for index time synonyms: fieldType name=text class=solr.TextField positionIncrementGap=100 omitNorms=false analyzer type=index filter class=solr.SynonymFilterFactory synonyms=synonyms.txt ignoreCase=true expand=true tokenizerFactory=KeywordTokenizerFactory/ charFilter class=solr.HTMLStripCharFilterFactory/ filter class=solr.WordDelimiterFilterFactory generateWordParts=1 generateNumberParts=1 catenateWords=1 catenateNumbers=1 catenateAll=0 / filter class=solr.StopFilterFactory ignoreCase=true words=stopwords.txt / filter class=solr.EnglishPorterFilterFactory protected=protwords.txt / filter class=solr.LowerCaseFilterFactory / filter class=solr.RemoveDuplicatesTokenFilterFactory / tokenizer class=solr.WhitespaceTokenizerFactory/ /analyzer .. And my synonyms.txt looks like this: floor locker=storage locker What am I doing wrong?
SpellCheck Help
I am trying to get the IndexBasedSpellChecker to work. I believe I have everything setup properly and the spellcheck component seems to be running but the suggestions list is empty. I am using SOLR 3.5 with Jetty. My solrconfig.xml and schema.xml are as follows: solrconfig.xml: http://pastie.org/private/z7sharm0ajlmm9hpy41v7g schema.xml: http://pastie.org/private/ykim99unbqfhumxxzbs6g
Re: SpellCheck Help
my copyField was defined as copyfield --- notice the lowercase f On Tue, Jan 10, 2012 at 2:50 PM, Dyer, James james.d...@ingrambook.comwrote: Three things to check: 1. Use a higher spellcheck.count than 1. Try 10. IndexBasedSpellChecker pre-filters the possibilities in a first pass of a 2-pass process. If spellcheck.count is too low, all the good suggestions might get filtered on the first pass and then it won't find anything on the second. 2. Be sure you're building the dictionary. Try adding spellcheck.build=true to your first query. You need to do do this every time you start the solr core. 3. Try a lower spellcheck.accuracy. Maybe the default .5 instead of the .7 you've got. One other thing to consider: - If the misspelled word exists in your index, the spellchecker won't try to correct it. This is true even if you're omitting words from the dictionary (for intance, by using thresholdTokenFrequency) James Dyer E-Commerce Systems Ingram Content Group (615) 213-4311 -Original Message- From: Donald Organ [mailto:dor...@donaldorgan.com] Sent: Tuesday, January 10, 2012 1:26 PM To: solr-user@lucene.apache.org Subject: SpellCheck Help I am trying to get the IndexBasedSpellChecker to work. I believe I have everything setup properly and the spellcheck component seems to be running but the suggestions list is empty. I am using SOLR 3.5 with Jetty. My solrconfig.xml and schema.xml are as follows: solrconfig.xml: http://pastie.org/private/z7sharm0ajlmm9hpy41v7g schema.xml: http://pastie.org/private/ykim99unbqfhumxxzbs6g
Re: Unable to get multicore working
When you go to /solr what do you see? On Tue, Aug 16, 2011 at 5:23 PM, David Sauve dnsa...@gmail.com wrote: I updated my `solr.xml` as follow: ?xml version=1.0 encoding=UTF-8 ? solr persistent=false cores adminPath=/admin/cores defaultCoreName=live core name=preview instanceDir=/home/webteam/config/search/preview dataDir=/home/webteam/preview/data / core name=staging instanceDir=/home/webteam/config/search/staging dataDir=/home/webteam/staging/data / core name=live instanceDir=/home/webteam/config/search/live dataDir=/home/webteam/live/data / /cores /solr and I'm still seeing the same 404 when I true to view /solr/admin/ or /solr/live/admin/ That said, the logs are showing a different error now. Excellent! The site schemas are loading! Looks like the site schemas have an issue: SEVERE: org.apache.solr.common.SolrException: Unknown fieldtype 'long' specified on field area_id Errr. Why would `long` be an invalid type? On Tuesday, 16 August, 2011 at 2:06 PM, Jaeger, Jay - DOT wrote: Whoops: That was Solr 4.0 (which pre-dates 3.1). I doubt very much that the release matters, though: I expect the behavior would be the same. -Original Message- From: Jaeger, Jay - DOT [mailto:jay.jae...@dot.wi.gov] Sent: Tuesday, August 16, 2011 4:04 PM To: solr-user@lucene.apache.org (mailto:solr-user@lucene.apache.org) Subject: RE: Unable to get multicore working I tried on my own test environment -- pulling out the default core parameter out, under Solr 3.1 I got exactly your symptom: an error 404. HTTP ERROR 404 Problem accessing /solr/admin/index.jsp. Reason: missing core name in path The log showed: 2011-08-16 16:00:12.469:WARN::/solr/admin/ java.lang.IllegalStateException: STREAM at org.mortbay.jetty.Response.getWriter(Response.java:616) at org.apache.jasper.runtime.JspWriterImpl.initOut(JspWriterImpl.java:187) at org.apache.jasper.runtime.JspWriterImpl.flushBuffer(JspWriterImpl.java:180) at org.apache.jasper.runtime.PageContextImpl.release(PageContextImpl.java:237) at org.apache.jasper.runtime.JspFactoryImpl.internalReleasePageContext(JspFactoryImpl.java:173) at org.apache.jasper.runtime.JspFactoryImpl.releasePageContext(JspFactoryImpl.java:124) (etc.) Adding the defaultCoreName fixed it. I expect this is indeed your problem. -Original Message- From: David Sauve [mailto:dnsa...@gmail.com] Sent: Tuesday, August 16, 2011 3:50 PM To: solr-user@lucene.apache.org (mailto:solr-user@lucene.apache.org) Subject: Re: Unable to get multicore working Nope. Only thing in the log: 1 [main] INFO org.mortbay.log - Logging to org.slf4j.impl.SimpleLogger(org.mortbay.log) via org.mortbay.log.Slf4jLog 173 [main] INFO org.mortbay.log - Redirecting stderr/stdout to /var/log/jetty/2011_08_16.stderrout.log On Tuesday, 16 August, 2011 at 1:45 PM, Alexei Martchenko wrote: Is your solr.xml in usr/share/jetty/solr/solr.xml? lets try this xml instead ?xml version=1.0 encoding=UTF-8 ? solr persistent=true cores adminPath=/admin/cores core name=core01 instanceDir=core01 / core name=core02 instanceDir=core02 / core name=core03 instanceDir=core03 / /cores /solr Can you see the logs? You should see something like this 16/08/2011 17:30:55 org.apache.solr.core.SolrResourceLoader init *INFO: Solr home set to 'solr/'* 16/08/2011 17:30:55 org.apache.solr.servlet.SolrDispatchFilter init INFO: SolrDispatchFilter.init() 16/08/2011 17:30:55 org.apache.solr.core.SolrResourceLoader locateSolrHome INFO: JNDI not configured for solr (NoInitialContextEx) 16/08/2011 17:30:55 org.apache.solr.core.SolrResourceLoader locateSolrHome *INFO: solr home defaulted to 'solr/' (could not find system property or JNDI)* 16/08/2011 17:30:55 org.apache.solr.core.CoreContainer$Initializer initialize *INFO: looking for solr.xml: usr/share/jetty/solr/solr.xml* 16/08/2011 17:30:55 org.apache.solr.core.SolrResourceLoader locateSolrHome INFO: JNDI not configured for solr (NoInitialContextEx) 16/08/2011 17:30:55 org.apache.solr.core.SolrResourceLoader locateSolrHome *INFO: solr home defaulted to 'solr/' (could not find system property or JNDI)* 16/08/2011 17:30:55 org.apache.solr.core.CoreContainer init *INFO: New CoreContainer: solrHome=solr/ instance=21357269* 16/08/2011 17:30:55 org.apache.solr.core.SolrResourceLoader init *INFO: Solr home set to 'solr/'* 16/08/2011 17:30:55 org.apache.solr.core.SolrResourceLoader init *INFO: Solr home set to 'solr\core01\'* 2011/8/16 David Sauve dnsa...@gmail.com (mailto:dnsa...@gmail.com) Just the one `solr.xml`. The one I added (well, symlinked form my config folder -- I like to keep my configurations files organized so they can be managed by git) `start.jar` is in `usr/share/jetty/start.jar`. On Tuesday, 16 August, 2011
