shubhamvishu commented on code in PR #16092:
URL: https://github.com/apache/lucene/pull/16092#discussion_r3278674064
##########
lucene/core/src/java/org/apache/lucene/codecs/lucene104/Lucene104ScalarQuantizedVectorsReader.java:
##########
@@ -212,7 +236,22 @@ public RandomVectorScorer getRandomVectorScorer(String
field, float[] target) th
fi.vectorDataOffset,
fi.vectorDataLength,
quantizedVectorData),
- target);
+ scoringTarget);
+ }
+
+ private HadamardRotation rotationFor(String field, int dimension) {
+ return rotations.computeIfAbsent(
+ field,
+ f ->
+ HadamardRotation.create(
+ dimension,
Lucene104ScalarQuantizedVectorsFormat.rotationSeed(f)));
+ }
+
+ private boolean isRotationEnabled(String field) {
+ FieldInfo info = fieldInfos.fieldInfo(field);
+ return info != null
+ && "true"
+
.equals(info.getAttribute(Lucene104ScalarQuantizedVectorsFormat.ROTATION_ENABLED_KEY));
Review Comment:
Yeah I also reasonate here. The initial version (1st commit) tried to retain
it in the sandox module -> then moved it into the main core Codec but it was
writing a random seed to the metadata of the field and I had to bump the codec
to 105. I honestly didn't wanted to bump the Codec for this given usecase so I
switched to use a constant seed and assuming rotation is enabled for all vector
fields by default(which obviously simplified all of this but takes away the
capability from user to configure it on per field basis). So eventually I
biased towards sharing the `rotationEnabled` flag (configured per vector field)
to query time via `FieldInfos` and avoiding Codec bump since this way we were
not breaking the backward compatibility and also how simple it was in nature.
I'm open to ideas whichever we would want to choose or if there is a better way
to share this info(rotation enabled/disabled) hopefully avoiding the codec bump.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
