mikemccand commented on issue #16029: URL: https://github.com/apache/lucene/issues/16029#issuecomment-4498414638
> > Implement random rotation of vectors and queries. This generates a more favorable value distribution for quantization and should help reclaim some of the accuracy losses in the first step. > > [@mccullocht](https://github.com/mccullocht) I have a working implementation solely for this with Claude's help (same Hadamard transform inspired by [@xande](https://github.com/xande) 's PR [#15903](https://github.com/apache/lucene/pull/15903) but in isolation) and ran some luceneutil tests on cohere v3 with rotation enabled. Here are results : Very cool! Is this (standalone random rotation) something we could merge in luceneutil? It's wild what this could unlock, what @mccullocht describes in OP. > * Implement residual vector quantization. After quantizing each vector `v`, quantize the residual `v - dequantize(quantize(v))` and write it to separate storage. This vector can be used to boost accuracy during reranking/rescoring or vector decoding while still allowing fast/cheap representations for graph traversal. This makes data blind quantization more palatable at low bit rates (1-2 bits/dim). At search time, how would we expose this? Somehow you need to use the original vector plus the quantized residuals to produce a more accurate vector for re-ranking. Maybe it'd have to be a dedicated reranking query that know about this Codec format... or maybe we could enhance `KnnVectorsFormat` API for first-class support, sort of a two-phased ranking API for vectors. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
