jerry-024 commented on PR #82: URL: https://github.com/apache/paimon-vector-index/pull/82#issuecomment-5505051844
Rewrote the branch to remove the root cause of the repeated stability blockers instead of patching them. The transposed-codebook kernel minimized direct squared differences while search scored codes via the `||q||² + ||c||² − 2q·c` expansion. Two different roundings of the same L2 distance meant add and search could disagree on argmin ties, and each stability certificate / exact fallback only covered the cases it was built for — @shyjsarah kept finding valid new ones because the inconsistency was structural. The new encoder computes distances with one SGEMM per sub-quantizer block using exactly the same identity as `encode_batch` and the search-time tables, so codes are byte-identical to the canonical per-vector path and the add/search inconsistency cannot occur. All the certificate/fallback machinery (`fvec_madd_two_minima`, per-sub exact fallback, serialized norm bounds) is deleted; the diff is now +386/−1. Fresh 8-thread benchmarks on the same M3 Pro (baseline = merge-base `b5b4a77`): 1.69x at 4096 rows (`d=768, m=192`), 2.16x at 32768 rows (`dsub=8`, which SGEMM handles natively rather than via scalar fallback), no small-batch regression below the 32-row threshold. Updated numbers are in the PR description. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
