liangkaiwen opened a new pull request, #4919: URL: https://github.com/apache/solr/pull/4919
https://issues.apache.org/jira/browse/SOLR-18424 <!-- _(If you are a project committer then you may remove some/all of the following template.)_ Before creating a pull request, please file an issue in the ASF Jira system for Solr: * https://issues.apache.org/jira/projects/SOLR For something minor (i.e. that wouldn't be worth putting in release notes), you can skip JIRA. To create a Jira issue, you will need to create an account there first. The title of the PR should reference the Jira issue number in the form: * SOLR-####: <short description of problem or changes> SOLR must be fully capitalized. A short description helps people scanning pull requests for items they can work on. Properly referencing the issue in the title ensures that Jira is correctly updated with code review comments and commits. --> # Description Solr 10 introduced Binary Quantization, however the recall drop-off is fairly sharp when used as-is. This can be partially offset by oversampling (retrieving more total documents from HNSW graph) and reranking (using raw non-quantized vector values from .vec). While reranking is possible through the standard Solr reranking feature, it is cumbersome syntactically and easy to get wrong. This aims to offer a cleaner and performant oversampling/reranking natively in the KNN query parser # Solution Code mostly written by Claude Opus 5 * Add a param 'rerankOversample' to the KNN query parser. This param accepts a positive integer and is optional (default value=1 when not provided). Behaviorally, this value is a multiplier on the existing topK * efSearch, and if it is > 1 do a rerank stage after search down to topK only. * Validation of rerankOversample. The provided value should be >= 1 and oversample should not be applied to BYTE encoded vectors * Uses generic RescoreTopNQuery and passes the similarity function explicitly, because the lucene rerank vector function can hit a NullPointerException when similarity function is not explicitly defined (as a binary quantized field definition might not). * **Latency / Recall Benchmarks** To be added soon # Tests * Manually querying * Unit tests * Benchmarking # Checklist Please review the following and check all that apply: - [ ] I have reviewed the guidelines for [How to Contribute](https://github.com/apache/solr/blob/main/CONTRIBUTING.md) and my code conforms to the standards described there to the best of my ability. - [ ] I have created a Jira issue and added the issue ID to my pull request title. - [ ] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended, not available for branches on forks living under an organisation) - [ ] I have developed this patch against the `main` branch. - [ ] I have run `./gradlew check`. - [ ] I have added tests for my changes. - [ ] I have added documentation for the [Reference Guide](https://github.com/apache/solr/tree/main/solr/solr-ref-guide) - [ ] I have added a [changelog entry](https://github.com/apache/solr/blob/main/dev-docs/changelog.adoc) for my change -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
