JingsongLi opened a new pull request, #8562: URL: https://github.com/apache/paimon/pull/8562
## Summary Builds the bucket-local maintenance layer for primary-key vector indexes on top of #8549. The maintainer restores active ANN state, reads complete vectors from compact data files, and produces index-file increments for compaction transitions without activating the write or query paths yet. ## Changes - Restore immutable bucket index state from active ANN payloads and validate vector field, index type, payload uniqueness, and one-to-one source-file coverage. - Read vector-only projections from physical compact data-file rows without deletion-vector filtering, preserving ordinals for later search-time filtering. - Maintain ANN payloads across compaction by removing segments that reference replaced files and rebuilding one multi-source segment for uncovered active sources. - Restrict eligible sources to `COMPACT` files above level 0, where rows are complete for deduplicate and partial-update tables. - Allow source-backed ANN segments to coexist in index manifests even though their ordinal ranges are segment-local and overlap. - Clean up newly created ANN payloads when a maintenance attempt fails. ## Testing - [x] Build a single-source ANN segment from a compact data file. - [x] Rebuild a multi-source ANN segment after partial compaction. - [x] Reject duplicate source coverage and mismatched vector index types during restore. - [x] Read vectors from a real Avro compact data file with vector-only projection while preserving NULL physical positions. - [x] Combine source-backed manifest entries with overlapping segment-local ordinal ranges. - [x] Run the focused Core suite: 14 tests passed. - [x] Run the non-fast Core reactor build with Checkstyle, Spotless, and Maven enforcers. ## Notes - Follows #8549, which introduced the primary-key vector index metadata and configuration foundation. - This PR intentionally does not wire the maintainer into the write lifecycle and does not add query routing. Those are follow-up changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
