goankur opened a new issue, #16902: URL: https://github.com/apache/lucene/issues/16902
### Description `Lucene104HnswScalarQuantizedVectorsFormat` quantizes FLOAT32 and FLOAT16 fields. BYTE fields go to the raw writer unchanged (`Lucene104ScalarQuantizedVectorsWriter#addField`), so the graph is built and searched over the full int8 vectors. At 4096 dims, that is 4 KiB per graph visit, and the whole of `.vec` must stay in RAM for the search to be fast. We propose to give BYTE fields the same two tiers that FLOAT16 fields already have: quantized codes for the graph search, and the raw vectors on disk for rescoring. This would be an opt-in format option, so existing BYTE fields, and segments written without codes, keep today's behavior. **Lucene does not quantize anything to int8 here.** The int8 vectors come from the caller: from an embedding model that outputs int8, or from the caller's own quantization before indexing. Lucene takes them as they are: - **Codes:** each vector is widened to float with `(float) b`, without a scale, and quantized like a float vector. Byte queries are widened the same way and scored against the codes. - **Rescoring:** the int8 query is scored against the raw int8 vectors with the existing int8 dot product. A new `FullPrecisionByteVectorSimilarityValuesSource` and a `byte[]` overload of `RescoreTopNQuery.createFullPrecisionRescorerQuery` do this, and they prefetch like the float path (#16705). The caller is responsible for two things: - **One scale for all vectors of the field.** Lucene stores no scale, so the int8 values must be comparable across vectors. A per-vector scale would break both the codes and the rescoring. - **The quality of the int8 vectors.** Lucene sees only the int8 values, so how they were produced is up to the caller. ### Relation to #16029 Part 3 of #16029 (residual quantization) gives float fields a precise rescoring tier without storing the floats. This proposal covers vectors that are already int8 when they reach Lucene. In that case there is no float input to quantize, so #16029 cannot help. The two do not overlap. @mccullocht, does this fit next to your plan? ### Results These results are on Amazon internal 4K dim embeddings: one index of hundreds of millions of vectors with 2 segments, on one m8gd.16xlarge (64 vCPU, 256 GiB RAM, local NVMe). The codes and the graph are in RAM, and the int8 vectors, several TB, are on the SSDs. We used 32 concurrent clients, with oversample 6 and ef_search 700: | | Recall@100 | QPS | |---|---|---| | Codes, then int8 rescoring | 0.950 | 1,909-1,925 | | Codes only | 0.733 | 2,938-2,941 | The codes alone cannot reach the target. A brute-force scan of all 1-bit codes gives recall@100 0.743, so recall at 0.95 needs the rescoring tier. The int8 tier is 2x smaller than fp16. In a small sample (19K vectors, 200 queries), its top-100 overlap with exact fp32 was 0.9976, against 0.9999 for fp16. ### Open questions 1. **Similarity.** Widening without a scale multiplies every vector by a large constant, so the float `DOT_PRODUCT` score, which expects unit vectors and clips at 0, is wrong. We used `MAXIMUM_INNER_PRODUCT`. Should the format reject other similarities for BYTE fields, or store a per-field scale? 2. **The merge graph.** Our prototype builds it from the raw int8 vectors by default, and from the codes when those vectors do not fit in RAM. Building from the int8 vectors reads `.vec` at random, and for our largest merge that was about 3x the RAM. A graph built from the int8 vectors needed 1-2% fewer visits per query than one built from the codes. Is a switch acceptable, or should merges always use the codes, as float fields do? The format also limits fields to 1024 dimensions (`getMaxDimensions`). We raised it with a format wrapper for these tests; that limit is out of scope here. We have a working prototype with tests, and can open a PR once the direction is agreed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
