slow-J commented on code in PR #16790:
URL: https://github.com/apache/lucene/pull/16790#discussion_r4454189430
##########
lucene/core/src/java/org/apache/lucene/util/quantization/OptimizedScalarQuantizer.java:
##########
@@ -192,25 +176,15 @@ public QuantizationResult scalarQuantize(
assert bits > 0 && bits <= 8;
float[] intervalScratch = new float[2];
int points = 1 << bits;
- double vecMean = 0;
- double vecVar = 0;
- float norm2 = 0;
- float centroidDot = 0;
- float min = Float.MAX_VALUE;
- float max = -Float.MAX_VALUE;
- for (int i = 0; i < vector.length; ++i) {
- if (similarityFunction != EUCLIDEAN) {
- centroidDot += vector[i] * centroid[i];
- }
- vector[i] = vector[i] - centroid[i];
- min = Math.min(min, vector[i]);
- max = Math.max(max, vector[i]);
- norm2 += (vector[i] * vector[i]);
- double delta = vector[i] - vecMean;
- vecMean += delta / (i + 1);
- vecVar += delta * (vector[i] - vecMean);
- }
- vecVar /= vector.length;
+ float[] centerStats = new float[5];
+ VectorUtil.osqCenter(vector, centroid, centerStats);
+ float centroidDot = centerStats[0];
+ float min = centerStats[1];
+ float max = centerStats[2];
+ float norm2 = centerStats[3];
+ double vecMean = centerStats[4] / (double) vector.length;
+ // Rounding can push the mean of squares minus the squared mean just below
zero.
+ double vecVar = Math.max(0, norm2 / (double) vector.length - vecMean *
vecMean);
Review Comment:
Thanks for validating and sorry for the churn here, I will bring back the
Welford in next commit!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
