mccullocht opened a new pull request, #16322: URL: https://github.com/apache/lucene/pull/16322
Adds StableTflSimilarity, a new similarity algorithm that estimates term rarity from term length and document length instead of corpus-level stats (doc frequency, total doc count, avg doc length). I presented this model at Haystack 2026 at Charlottesville (slides). It keeps the same overall shape as BM25 - a sum over query terms of a term-frequency saturation factor times a per-term rarity weigh, but swaps IDF for a synthetic, corpus-independent term rarity (TR) term, computed as the probability that a given term appears in a doc at least once based only on the term length and doc length. Backport of the original PR but needs some non-trivial integration that could use review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
