LuciferYang opened a new issue, #9895: URL: https://github.com/apache/paimon/issues/9895
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Paimon version master ### Compute Engine Any (primary-key table vector search) ### Minimal reproduce step On a `first-row` merge-engine table with deletion vectors disabled and a `pk-vector` index, write a key, let it compact, then write the same key again (a version `first-row` drops on read). Run a vector search with `vector-index.search-mode` set to `full` or `detail`. ### What doesn't meet your expectations? The vector search can return the superseded row version, which a normal merge-on-read query would never return. Root cause: the exact scan in `PrimaryKeyVectorBucketSearch` returns physical row positions per file with no dedup by primary key, and the readback (`PrimaryKeyIndexedSplitRead` / `RawFileSplitRead`) reads those positions directly without a merge-on-read pass. `first-row` resolves duplicate keys only through the merge/compaction path, so when a superseded version has not yet been merged away it is physically present and the exact scan can surface it. This is specific to the `first-row` engine with deletion vectors disabled; deletion-vector-enabled tables mark superseded rows and exclude them from the exact scan. This predates #9894. That change extends the exact scan to level-0 APPEND files, which makes the surfacing more frequent (a fresh superseded write is the common case), but the underlying gap already applied to superseded versions promoted into compacted files. ### Anything else? One direction would be to apply a primary-key dedup to the exact-scan candidates before ranking (or require deletion vectors for `full`/`detail` on primary-key vector search). Filed for tracking, not blocking #9894. ### Are you willing to submit a PR? - [ ] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
