JingsongLi opened a new pull request, #35: URL: https://github.com/apache/paimon-vector-index/pull/35
## Summary This PR tightens the v1 storage metadata for IVF-HNSW formats before release, making the on-disk contract more explicit and easier to validate. It also hardens IVF-PQ size and offset calculations to fail early on invalid or overflowing metadata. ## Changes - Add required IVF-HNSW v1 format flags for raw id encoding and graph-v1 adjacency encoding, and reject missing or unknown flags when reading. - Replace the HNSW offset-table reserved field with `payload_bytes_len`, then validate payload lengths against the expected graph payload size. - Validate IVF-HNSW-SQ header SQ bounds summaries against the global SQ bounds section. - Use checked conversions and arithmetic for IVF-PQ dimensions, vector counts, offsets, and list sizes. - Update the storage-format documentation and HNSW golden fixture hex files for the tightened v1 layout. ## Testing - [x] `cargo test -p paimon-vindex-core --test storage_format_fixtures` - [x] `cargo test -p paimon-vindex-core` - [x] `git diff --check` ## Notes This is intentionally a v1-only format tightening before release; no legacy layout compatibility is added. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
