JingsongLi opened a new pull request, #8563: URL: https://github.com/apache/paimon/pull/8563
## Summary Activates primary-key vector index maintenance in the write path on top of #8549 and #8562. Complete Level 1 compact files become ANN sources, and bucket-local ANN additions and removals are published with the same compact increment as their source data transition. ## Changes - Force lookup compaction to rewrite Level 0 files for both deduplicate and partial-update tables, ensuring vector indexes are built only from complete Level 1 rows. - Create and restore a bucket-local vector index maintainer for primary-key table writers using the configured vector field, metric, algorithm, and field-scoped index options. - Publish ANN additions and removals through the matching compact increment and retain maintainer state across writer checkpoints. - Restore active source-backed ANN metadata from filesystem manifests without requiring an auxiliary directory index. - Extend Flink coordinated writer restore requests and responses to round-trip active vector index payloads while preserving existing constructors. ## Testing - [x] Verify deduplicate and partial-update lookup compaction both rewrite Level 0 files. - [x] Verify vector index additions and removals are attached to the same compact data transition. - [x] Verify filesystem writer restore returns active source-backed ANN metadata. - [x] Verify Flink coordinator restore round-trips ANN metadata with inclusive ordinal ranges. - [x] Run the related Core suite: 19 tests passed. - [x] Run the focused Flink coordinator test: 1 test passed. - [x] Run non-fast Core/Flink compilation with Checkstyle, Spotless, and Maven enforcers. ## Notes - Follows #8562, which introduced bucket-level primary-key vector index maintenance. - This PR intentionally covers only write publication and restore. Primary-key vector query routing remains a follow-up. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
