JingsongLi commented on code in PR #82:
URL:
https://github.com/apache/paimon-vector-index/pull/82#discussion_r3851077711
##########
core/src/ivfpq.rs:
##########
@@ -228,9 +293,16 @@ impl IVFPQIndex {
// L2/IP without OPQ borrows the caller's batch instead of copying it.
let processed = self.preprocess_queries(data, n);
- let assignments =
- kmeans::find_nearest_batch(&processed, n,
&self.quantizer_centroids, self.nlist, d);
-
+ // Both branches return the exact nearest centroid; the projection only
+ // prunes the scan (see `projected_assign`).
+ let assignments = match &self.coarse_projection {
Review Comment:
[P2] Invalidate the projection when centroids change
`quantizer_centroids` remains publicly mutable, so an existing low-level
caller can replace centroids after `train()`. Before this change, `add` scanned
the current field; this branch instead uses bounds cached from the old
centroids and can prune the new nearest list. In a deterministic repro
(`d=64,nlist=512`, Auto projection), replacing centroid 500 with `centroid[0] +
0.1` and then adding that vector assigns it to list 0 (direct L2 about 0.64)
instead of list 500 (distance 0), and residual encoding also uses the wrong
centroid. Please make centroid replacement invalidate/rebuild
`coarse_projection` (or make the centroids immutable behind an updating API),
and add a mutate-after-train regression.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
