[
https://issues.apache.org/jira/browse/LUCENE-8010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16306129#comment-16306129
]
Adrien Grand commented on LUCENE-8010:
--------------------------------------
I suspect we fixed at least some of those since I could run the entire test
suite 3 times without scoring-related issues after enabling AxiomaticF3 and
BooleanSimilarity. I'll watch the build.
> fix or sandbox similarities in core with problems
> -------------------------------------------------
>
> Key: LUCENE-8010
> URL: https://issues.apache.org/jira/browse/LUCENE-8010
> Project: Lucene - Core
> Issue Type: Bug
> Reporter: Robert Muir
> Attachments: LUCENE-8010.patch
>
>
> We want to support scoring optimizations such as LUCENE-4100 and LUCENE-7993,
> which put very minimal requirements on the similarity impl. Today
> similarities of various quality are in core and tests.
> The ones with problems currently have warnings in the javadocs about their
> bugs, and if the problems are severe enough, then they are also disabled in
> randomized testing too.
> IMO lucene core should only have practical functions that won't return
> {{NaN}} scores at times or cause relevance to go backwards if the user's
> stopfilter isn't configured perfectly. Also it is important for unit tests to
> not deal with broken or semi-broken sims, and the ones in core should pass
> all unit tests.
> I propose we move the buggy ones to sandbox and deprecate them. If they can
> be fixed we can put them back in core, otherwise bye-bye.
> FWIW tests developed in LUCENE-7997 document the following requirements:
> * scores are non-negative and finite.
> * score matches the explanation exactly.
> * internal explanations calculations are sane (e.g. sum of: and so on
> actually compute sums)
> * scores don't decrease as term frequencies increase: e.g. score(freq=N +
> 1) >= score(freq=N)
> * scores don't decrease as documents get shorter, e.g. score(len=M) >=
> score(len=M+1)
> * scores don't decrease as terms get rarer, e.g. score(term=N) >=
> score(term=N+1)
> * scoring works for floating point frequencies (e.g. sloppy phrase and
> span queries will work)
> * scoring works for reasonably large 64-bit statistic values (e.g.
> distributed search will work)
> * scoring works for reasonably large boost values (0 .. Integer.MAX_VALUE,
> e.g. query boosts will work)
> * scoring works for parameters randomized within valid ranges
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
