wangyong9999 opened a new issue, #9458: URL: https://github.com/apache/paimon/issues/9458
### Paimon version `master` at `052adc366f63adc613dce01e2f1e7fec82f3c449`. ### Compute Engine Java core scan planning. The issue is engine-independent and affects readers using the source-backed primary-key sorted index plan. ### Minimal reproduce step 1. Build one primary-key sorted-index payload for two compacted source files at the same positive data level. The payload metadata keeps the ordered source group `[A, B]` and its row positions use that group-wide ordinal space. 2. Advance to a snapshot where `A` has been retired while `B` remains active at the same data level. Keep the payload active because it still covers `B`. 3. Plan an indexed predicate scan for `B`. `PkSortedBucketIndexState.fromActiveDataFiles` currently compares the current active level sources `[B]` with the immutable payload sources `[A, B]` using exact list equality. It rejects the payload, so `B` falls back to a raw scan. ### What doesn't meet your expectations? Retiring one source file should not invalidate index coverage for the other active files in the same immutable payload group. The reader should: - retain the payload's complete ordered source list for group-ordinal localization; - accept the payload when it has a non-empty intersection with eligible active files at the metadata-declared level and matching file row counts; - mark only that active intersection as covered; - leave newly active files not listed by the payload uncovered so they are scanned normally; and - continue to reject malformed, ambiguous, wrong-level, or zero-active-source payloads. This changes neither the persisted metadata format nor writer behavior. Apache Paimon C++ has the same reader-side issue and the corresponding fix is under review in [apache/paimon-cpp#253](https://github.com/apache/paimon-cpp/pull/253). I searched existing issues and pull requests and found no report of this lifecycle case. ### Are you willing to submit a PR? Yes. I will submit a focused fix with state-planning and row-offset localization tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
