Pranshu-S commented on code in PR #16764:
URL: https://github.com/apache/lucene/pull/16764#discussion_r4186002608
##########
lucene/core/src/java/org/apache/lucene/codecs/lucene104/Lucene104ScalarQuantizedVectorsWriter.java:
##########
@@ -642,7 +643,7 @@ static int calculateCentroid(MergeState mergeState,
FieldInfo fieldInfo, float[]
for (int i = 0; i < mergeState.knnVectorsReaders.length; i++) {
KnnVectorsReader knnVectorsReader = mergeState.knnVectorsReaders[i];
if (knnVectorsReader == null) continue;
- count += accumulateCentroid(knnVectorsReader, fieldInfo, centroid);
+ count += accumulateCentroid(knnVectorsReader, fieldInfo,
mergeState.liveDocs[i], centroid);
Review Comment:
Right, looks like this comment needs to be removed now. We fixed two bugs in
one go :)
From what I understand, needsMergeScorer is a predicate passed as
[Lucene99HnswVectorsWriter::buildsGraph](https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/codecs/lucene99/Lucene99HnswVectorsWriter.java#L711-L713),
which checks whether the vector count is high enough to require an HNSW graph.
If it is, Lucene prepares [query-side
data](https://github.com/apache/lucene/blob/3970208e0d4d5e997a10baac816bd7d1d7693350/lucene/core/src/java/org/apache/lucene/codecs/lucene104/Lucene104ScalarQuantizedVectorsWriter.java#L374-L376)
to score vectors during graph construction.
Previously, it could receive a count including deleted vectors, causing
unnecessary query-side data preparation even when the live-vector count was
below the threshold.
The actual graph-building decision already uses
[LiveDocs](https://github.com/apache/lucene/blob/14dacef992690d37d04a7ad816f534835ee1fffe/lucene/core/src/java/org/apache/lucene/codecs/lucene99/Lucene99HnswVectorsWriter.java#L518)
from the merged segment, which only includes live vectors. With this change,
both decisions use the live-vector count, avoiding unnecessary preparation
without changing whether a graph is built.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
