XiaoHongbo-Hope opened a new pull request, #8946:
URL: https://github.com/apache/paimon/pull/8946
### Purpose
On a deletion-vector (DV) enabled table, a vector search plans its
global-index
scan at one snapshot but resolved the live-row filter and downstream reads
at the
latest snapshot. A `materialize-deletion` compaction renumbers row-ids (drops
DV-deleted rows and rewrites files). If one commits between plan and read,
the
index's row-ids (numbering A) no longer map to the data (numbering B), so the
search returns wrong or missing rows.
This is latent on master today (index build on DV tables is currently
blocked),
but becomes reachable once building Lumina indexes on DV tables is allowed.
It is
a self-contained read-path fix and is submitted independently.
### Change
- Add `VectorScan.Plan.snapshot()` (default `null`) carrying the snapshot
the plan
was built against.
- Pin `GlobalIndexLiveRowFilter` and `DataEvolutionGlobalIndexScanner` to
that
snapshot so index scan, live-row filtering, and read all observe one
consistent
row-id space.
- Thread the pinned snapshot through the data-evolution and Flink read
drivers.
### Tests
`VectorSearchBuilderTest#testVectorSearchPinsLiveRowFilterToPlanSnapshot`:
plans at
S0, commits deletion vectors to reach S1, then reads with the S0 plan and
asserts
the row survives. Fails without the pin.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
