LuciferYang opened a new pull request, #9896: URL: https://github.com/apache/paimon/pull/9896
### Purpose close #9894 `PrimaryKeyVectorRead` handed the bucket search only the files `PrimaryKeyIndexSourcePolicy.shouldRead` accepts (complete COMPACT level>0 outputs), so Level-0 APPEND files, the rows written since the last compaction, were omitted by every search mode, including `full` and `detail`, which are documented to exact-scan unindexed data files. This passes all data files of the split (`filesToSearch`) to the bucket search instead. The exact fallback then covers the files the ANN segments do not, `FAST` keeps ignoring them, and the ANN state builder (`PkVectorBucketIndexState.fromActiveDataFiles`) still applies the compact-file policy internally, so the ANN segment state is unchanged and only the exact-scan file set widens. Note: on a `first-row` merge-engine table with deletion vectors disabled, exact-scanning uncompacted files in `full`/`detail` mode can surface a superseded row version, because the exact scan returns physical rows without a merge-on-read dedup. This is a pre-existing property of the exact scan (it already applied to superseded versions promoted into compacted files); including level-0 files makes it more frequent rather than introducing it. The deletion-vector-enabled path is unaffected, since superseded rows are deletion-vector-marked and excluded. Tracked in #9895. ### Tests The synchronous write harness compacts eagerly and cannot hold a stable level-0 window, so the coverage pins the two halves of the chain separately: - `PrimaryKeyVectorReadTest#testFilesToSearchIncludesAppendLevel0Files`: `filesToSearch` returns an APPEND level-0 file that `shouldRead` rejects. Against the old filtered list the append file is absent, so this fails. - `PrimaryKeyVectorReadTest#testFullModeBucketSearchExactScansAppendLevel0Files`: the bucket search exact-scans that level-0 file in `full` mode and returns it, while `FAST` still skips it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
