goankur opened a new pull request, #16771: URL: https://github.com/apache/lucene/pull/16771
### Description Fast-follow to #16705. Rescoring with full-precision vectors reads one whole vector per candidate. A `1024-dim` float32 vector is exactly `4096 bytes`, but `Lucene99FlatVectorsWriter` aligns the vector data section to `64 bytes`. Every vector therefore straddles two pages, and every rescoring read touches one more page than it needs. This changes the float32 alignment to 4096 bytes. That is still a multiple of 64, so the alignment that suits Arm Neoverse scoring is kept. This is not a format change. Readers locate the data through the offset stored in `.vem`, so existing indexes open unchanged. The cost is less than one page of padding per field per segment. ### Measurements Same setup and methodology as the #16705 table: - Dataset: [CohereLabs/msmarco-v2.1-embed-english-v3](https://huggingface.co/datasets/CohereLabs/msmarco-v2.1-embed-english-v3) - AWS `g6.4xl`: 64 GB RAM, 16 vCPU, 1x600 GB NVMe SSD - 100M vectors, 1-bit BBQ with float32 rescoring (405 GB index), `maxConn=64`, `beamWidth=250`, 17 segments - Open-loop Poisson at 280 QPS, 16 searcher threads - Binary codes, HNSW graph and vector metadata preloaded - Recall from a preceding 50K-query warm-up pass; the same queries are replayed for the latency and I/O metrics - Both arms use the same build: `main` plus this change, with the `MemorySegmentIndexInput#prefetch` backoff bypassed (see #16145) - Both arms use the same index. The baseline was produced by rewriting each segment 1:1 through `main`'s codec, so segment count, HNSW graphs and documents are identical and only the padding differs. Metric | Baseline (64 B) | Candidate (4 KiB) | delta -- | -- | -- | -- recall | 0.9601 | 0.9601 | identical achieved QPS | 281.3 | 281.4 | 0 shed | 0.000% | 0.000% | 0 mean | 45.48 ms | 30.10 ms | −33.8% p99 | 245.08 ms | 53.54 ms | −78.2% p999 | 285.87 ms | 73.91 ms | −74.1% max | 309.30 ms | 109.79 ms | −64.5% aqu-sz | 32.29 | 13.22 | −59.1% rareq-sz | 6.96 kB | 4.01 kB | −42.4% MB/s | 775.9 | 386.8 | −50.1% IOPS | 114,090 | 98,813 | −13.4% r_await | 0.283 ms | 0.134 ms | −52.7% The average read drops to exactly one page, and the device moves half the bytes for the same queries. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
