iverase commented on PR #15732: URL: https://github.com/apache/lucene/pull/15732#issuecomment-4011978065
In this new iteration I introduced the methods KnnVectorsWriter#mergeFlatVectors and KnnVectorsWriter#mergeVectorIndex. The first method is called for all the fields first and after that we call the method mergeVectorIndex for all the fields again. One important side efect of this work is that we need to make sure that calls to `PerFieldKnnVectorsFormat#getKnnVectorsFormatForField` is idempotent. I hit a bug in luceneutil where we are creating a new instance every time this method was called. With respect the current implementation, we are now only writing temporary files for asymetric quantizations where we want to store the query bits on a file as it is not part of the index. The trick to make it work is to move the method `FlatVectorsWriter#mergeOneFieldToIndex` to `QuantizeVectorReader#getRandomVectorScorerSupplierForMerge`. It is in this method where we create the tmp files for asymetric quatization. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
