benwtrent opened a new pull request, #15607: URL: https://github.com/apache/lucene/pull/15607
This adds bulk scoring to diversity check. While this means that diversity check cannot exit super early (e.g. if it only needs to check 2 docs), I continually see diversity check as being the most expensive part of HNSW graph merging. This tells me that typically, it isn't just one doc that is checked. I ran 1M 768 cohere, force-merging with 4 threads. baseline: 128.01 candidate: 92.15 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
