benwtrent commented on code in PR #15732:
URL: https://github.com/apache/lucene/pull/15732#discussion_r3029428043
##########
lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsWriter.java:
##########
@@ -96,19 +116,30 @@ public final void merge(MergeState mergeState) throws
IOException {
}
}
+ // Phase 1: merge flat vectors for all fields, collecting deferred work
+ List<IORunnable> deferredWork = new ArrayList<>();
for (FieldInfo fieldInfo : mergeState.mergeFieldInfos) {
if (fieldInfo.hasVectorValues()) {
if (mergeState.infoStream.isEnabled("VV")) {
mergeState.infoStream.message("VV", "merging " +
mergeState.segmentInfo);
}
- mergeOneField(fieldInfo, mergeState);
+ IORunnable deferred = mergeOneField(fieldInfo, mergeState);
+ if (deferred != null) {
+ deferredWork.add(deferred);
+ }
if (mergeState.infoStream.isEnabled("VV")) {
mergeState.infoStream.message("VV", "merge done " +
mergeState.segmentInfo);
}
}
}
+
+ // Phase 2: execute deferred work (e.g., graph construction using the
written flat vectors)
+ for (IORunnable runnable : deferredWork) {
+ runnable.run();
+ }
Review Comment:
lets add info streams around this as well.
##########
lucene/core/src/java/org/apache/lucene/codecs/lucene99/Lucene99HnswVectorsReader.java:
##########
@@ -434,6 +436,16 @@ public ScalarQuantizer getQuantizationState(String field) {
return null;
}
+ @Override
+ public CloseableRandomVectorScorerSupplier
getRandomVectorScorerSupplierForMerge(
+ FieldInfo fieldInfo, SegmentWriteState segmentWriteState) throws
IOException {
+ if (flatVectorsReader instanceof QuantizedVectorsReader) {
+ return ((QuantizedVectorsReader) flatVectorsReader)
Review Comment:
```suggestion
if (flatVectorsReader instanceof QuantizedVectorsReader
flatVectorsReader) {
return flatVectorsReader
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
