Adrien Grand resolved LUCENE-8010.
    Resolution: Fixed

> fix or sandbox similarities in core with problems
> -------------------------------------------------
>                 Key: LUCENE-8010
>                 URL: https://issues.apache.org/jira/browse/LUCENE-8010
>             Project: Lucene - Core
>          Issue Type: Bug
>            Reporter: Robert Muir
>            Priority: Major
>             Fix For: master (8.0)
>         Attachments: LUCENE-8010.patch
> We want to support scoring optimizations such as LUCENE-4100 and LUCENE-7993, 
> which put very minimal requirements on the similarity impl. Today 
> similarities of various quality are in core and tests. 
> The ones with problems currently have warnings in the javadocs about their 
> bugs, and if the problems are severe enough, then they are also disabled in 
> randomized testing too.
> IMO lucene core should only have practical functions that won't return 
> {{NaN}} scores at times or cause relevance to go backwards if the user's 
> stopfilter isn't configured perfectly. Also it is important for unit tests to 
> not deal with broken or semi-broken sims, and the ones in core should pass 
> all unit tests.
> I propose we move the buggy ones to sandbox and deprecate them. If they can 
> be fixed we can put them back in core, otherwise bye-bye.
> FWIW tests developed in LUCENE-7997 document the following requirements:
>    * scores are non-negative and finite.
>    * score matches the explanation exactly.
>    * internal explanations calculations are sane (e.g. sum of: and so on 
> actually compute sums)
>    * scores don't decrease as term frequencies increase: e.g. score(freq=N + 
> 1) >= score(freq=N)
>    * scores don't decrease as documents get shorter, e.g. score(len=M) >= 
> score(len=M+1)
>    * scores don't decrease as terms get rarer, e.g. score(term=N) >= 
> score(term=N+1)
>    * scoring works for floating point frequencies (e.g. sloppy phrase and 
> span queries will work)
>    * scoring works for reasonably large 64-bit statistic values (e.g. 
> distributed search will work)
>    * scoring works for reasonably large boost values (0 .. Integer.MAX_VALUE, 
> e.g. query boosts will work)
>    * scoring works for parameters randomized within valid ranges

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Reply via email to