[
https://issues.apache.org/jira/browse/LUCENE-8010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16303846#comment-16303846
]
Adrien Grand commented on LUCENE-8010:
--------------------------------------
I'd like to get these changes in so that we can reenable tests on these
similarities. I'll merge later this week if there are no objections, we can
still look into improving the hacks later if this hurts relevance?
> fix or sandbox similarities in core with problems
> -------------------------------------------------
>
> Key: LUCENE-8010
> URL: https://issues.apache.org/jira/browse/LUCENE-8010
> Project: Lucene - Core
> Issue Type: Bug
> Reporter: Robert Muir
> Attachments: LUCENE-8010.patch
>
>
> We want to support scoring optimizations such as LUCENE-4100 and LUCENE-7993,
> which put very minimal requirements on the similarity impl. Today
> similarities of various quality are in core and tests.
> The ones with problems currently have warnings in the javadocs about their
> bugs, and if the problems are severe enough, then they are also disabled in
> randomized testing too.
> IMO lucene core should only have practical functions that won't return
> {{NaN}} scores at times or cause relevance to go backwards if the user's
> stopfilter isn't configured perfectly. Also it is important for unit tests to
> not deal with broken or semi-broken sims, and the ones in core should pass
> all unit tests.
> I propose we move the buggy ones to sandbox and deprecate them. If they can
> be fixed we can put them back in core, otherwise bye-bye.
> FWIW tests developed in LUCENE-7997 document the following requirements:
> * scores are non-negative and finite.
> * score matches the explanation exactly.
> * internal explanations calculations are sane (e.g. sum of: and so on
> actually compute sums)
> * scores don't decrease as term frequencies increase: e.g. score(freq=N +
> 1) >= score(freq=N)
> * scores don't decrease as documents get shorter, e.g. score(len=M) >=
> score(len=M+1)
> * scores don't decrease as terms get rarer, e.g. score(term=N) >=
> score(term=N+1)
> * scoring works for floating point frequencies (e.g. sloppy phrase and
> span queries will work)
> * scoring works for reasonably large 64-bit statistic values (e.g.
> distributed search will work)
> * scoring works for reasonably large boost values (0 .. Integer.MAX_VALUE,
> e.g. query boosts will work)
> * scoring works for parameters randomized within valid ranges
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
