benwtrent commented on PR #15722:
URL: https://github.com/apache/lucene/pull/15722#issuecomment-3927644203
I don't mind the interface. But it does seem that the "scorer.bulkScore"
should "just do the right thing" and prefetch if necessary.
I think this will harm the "preferred path", which is the one where all
vectors are in MMAP'd space. We need to benchmark with quantized vectors, where
the vector ops during traversal are magnitudes cheaper, not just floating point.
In the aggressively nasty path, I am not sure prefetching 16-32 vectors at a
time will give us much as when it comes to modern IO throughput numbers, that
is barely scratching the surface (4kb * 16 if using raw vectors, which nobody
should be using raw vectors anymore...way lower memory numbers if
quantized...). If we are prefetching during graph traversal due to not having
enough memory, I think we need to prefetch WAY more aggressively. Like,
prefetching multiple candidate neighbors in the future (even ones we won't end
up scoring :/).
All in all, I think if we want HNSW to act nice in low memory, we need to:
- invest more in the Bipartite ordering of vectors so that neighbors are
near each other on disk
- Prefetch larger blocks hoping to get many vectors we care about besides
the immediate ordinals.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
