mccullocht opened a new pull request, #16685: URL: https://github.com/apache/lucene/pull/16685
Add a new Mode option to scalar quantization that allows switching between 3 different storage representations: CENTERED maintains the existing path: a mean vector is computed per-segment and every vector is quantized as a residual against this mean vector. DATA_BLIND_WITH_FLOATS skips computing a mean vector and provides a zero vector instead. This makes merging cheaper -- there's no need to compute the segment mean vector or re-quantize the contents of the segment. The original input float vector is retained. DATA_BLIND_WITHOUT_FLOATS extends the approach above by dropping the float vectors as well. This only support for symmetric quantization; asymmetric quantization needs to re-read the original floats as "queries" for graph search. Both CENTERED and DATA_BLIND_WITH_FLOATS require that there is an input float field on segment inputs or merges are failed, when floats are dropped we allow inputs with he same encoding. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
