mccullocht commented on PR #15722: URL: https://github.com/apache/lucene/pull/15722#issuecomment-3929132869
> I don't mind the interface. But it does seem that the "scorer.bulkScore" should "just do the right thing" and prefetch if necessary. Agreed, I think we'd like prefetching to be internal to the scorer. > I think this will harm the "preferred path", which is the one where all vectors are in MMAP'd space. We need to benchmark with quantized vectors, where the vector ops during traversal are magnitudes cheaper, not just floating point. Is this ultimately a result of the behavior of mmap input `prefetch()`? The currently implementation does a synchronous `madvise()` syscall which I would expect to harm performance when everything is in memory and may be worse than doing a read when the data has spilled to storage. > If we are prefetching during graph traversal due to not having enough memory, I think we need to prefetch WAY more aggressively. Like, prefetching multiple candidate neighbors in the future (even ones we won't end up scoring :/). Popping multiple candidates in the graph searcher like this is the suggested approach for parallelizing IO in the DiskANN paper. This would be a another query parameter unless we can reliably detect low memory conditions which is challenging if you let mmap manage caching :/. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
