Hi Derek,
the BQ (Boost Query) parameter is an additive contribution to the score.
Solr calculates the score contribution of each BQ query and adds it to the
score calculated so far.
*bq=P_NewShortDescription:mask^-1 *
This is not supported: boost must be a positive float, got -1.0
org.apache.lucene.search.BoostQuery#BoostQuery
I thought to achieve this effect you could just use boost function queries.
For example for an additive negative contribution:
bf=mul(query($qq),-10)&qq=P_NewShortDescription:mask
But the score contribution is zero :
0.0 = FunctionQuery(product(query(title:cities,def=0.0),const(-10))),
product of:\n -32.01146 =
product(query(title:cities,def=0.0)=3.201146,const(-10))\n 1.0 = boost\n"
So I suspect for additive modifications to the score you have also some
code check that prevents that internally.
The same if your multiplicative :
boost=mul(query($qq),-10)&qq=P_NewShortDescription:mask
0 = truncated score, max of:
0.0 = minimum score
-298.690542362805 = product of:
9.330738 = sum of:
...
Playing a bit with boost function you should be able to get your effect
anyway.
*bq=(*:* -P_NewShortDescription:"mask")^0.5*
Using this approach you are boosting up all documents that don't contain
the term 'mask' in such field.
Fine-tuning the amount of boost this works.
*Classic Question*
*The classic question remains: how much shoud I boost? (negative or
positive)*
You could improve one query and break 10 others.
The recommendation here is:
set up a search quality evaluation framework to measure your favourite
metrics across multiple queries
We contributed Rated Ranking Evaluator to the Open Source community, a
powerful tool to measure search quality, check it out, it could help you:
https://github.com/SeaseLtd/rated-ranking-evaluator
For enterprise needs, we have also developed RRE-Enterprise( coming
soon: https://sease.io/2019/12/road-to-rated-ranking-evaluator-enterprise.html
)
Cheers
--------------------------
Alessandro Benedetti
Apache Lucene/Solr Committer
Director, R&D Software Engineer, Search Consultant
www.sease.io
On Tue, 13 Apr 2021 at 03:15, Derek Poh <[email protected]>
wrote:
> Hi
>
> I am trying to "de-boost" documents that have certain word(s) so that
> they are pushed back to the end of the result set. Or at least not in
> the first 60 records of the result set.
>
> The query is doing a q=*:* with a few filter query (fq) parameters.
>
> This is an example of the query which uses bq to check for the word
> "mask" and give it a negative boost factor,
>
>
> http://localhost:8983/solr/product/select?start=0&rows=140&fq=P_DO_Flag:(%22Y%22)&fq=P_CategoryGroupId:(11044)&fq=-(P_SPC_Flag:(%22Y%22))&q=*:*&sort=score%20desc,P_DO_SPPRank%20asc,P_DO_Coupon_Flag%20desc,P_SupplierRanking%20desc,random_515%20asc&defType=edismax&bq=P_NewShortDescription:mask
> ^-1&wt=xml&fl=P_ProductId,P_VeryShortDescription,score,P_DO_Flag,P_NewShortDescription
>
> I have tried with
> bq=P_NewShortDescription:mask^-1
>
> bq=(*:* -P_NewShortDescription:"mask")^0.5
>
> But the documents that have the word "mask" are not de-boost or pushed
> back to the back of the result set.
>
> In the result of the query example above, there are 3 documents that
> have the word "Mask" in the field P_NewShortDescription.
> 2 of them are not de-boost as they have the same score of 1.5 as those
> documents that do not have "mask" in them.
> The remaining 1 document does have a lower score of 1 and it is at the
> end of the result set.
>
> Could you advice how to correctly de-boost documents that have certain
> word(s) in them?
>
> This is the definiton of the field that is checked on.
> <field name="P_NewShortDescription" type="text_general" omitNorms="true"
> multiValued="false" indexed="true" stored="true"/>
>
> <fieldType name="text_general" class="solr.TextField"
> positionIncrementGap="100">
> <analyzer type="index">
> <tokenizer class="solr.StandardTokenizerFactory"/>
> <filter class="solr.LowerCaseFilterFactory"/>
> <filter class="solr.StopFilterFactory" words="stopwords.txt"
> ignoreCase="true"/>
> <filter class="solr.KeywordMarkerFilterFactory"
> protected="protwords.txt"/>
> <filter class="solr.SynonymGraphFilterFactory" expand="true"
> ignoreCase="true" synonyms="synonyms.txt"/>
> <filter class="solr.KStemFilterFactory"/>
> </analyzer>
> <analyzer type="query">
> <tokenizer class="solr.StandardTokenizerFactory"/>
> <filter class="solr.LowerCaseFilterFactory"/>
> <filter class="solr.StopFilterFactory" words="stopwords.txt"
> ignoreCase="true"/>
> <filter class="solr.KeywordMarkerFilterFactory"
> protected="protwords.txt"/>
> <filter class="solr.SynonymGraphFilterFactory" expand="true"
> ignoreCase="true" synonyms="synonyms.txt"/>
> <filter class="solr.KStemFilterFactory"/>
> </analyzer>
> </fieldType>
>
> Solr version = 7.7.2
>
> Regards,
> Derek
>
> ----------------------
> CONFIDENTIALITY NOTICE
>
> This e-mail (including any attachments) may contain confidential and/or
> privileged information. If you are not the intended recipient or have
> received this e-mail in error, please inform the sender immediately and
> delete this e-mail (including any attachments) from your computer, and you
> must not use, disclose to anyone else or copy this e-mail (including any
> attachments), whether in whole or in part.
>
> This e-mail and any reply to it may be monitored for security, legal,
> regulatory compliance and/or other appropriate reasons.
>
>
