JingsongLi opened a new issue, #1050: URL: https://github.com/apache/paimon-rust/issues/1050
### Problem PyPaimon currently falls back to the Python reader when projecting `_SEQUENCE_NUMBER` over partial Data Evolution files. Rust only registers providers for physical user columns and cannot provide the non-null tracking sequence from normal-file metadata when that column is absent. Sequence predicates also need explicit metadata handling. Their placeholder positions must not resolve to unrelated user/partition columns or file statistics, and exact evaluation must happen after metadata assignment or primary-key merging. Reading only a user key plus sequence can otherwise prune the newest partial normal file needed for the version. For primary-key aggregation and partial update, sequence belongs to the resulting KV record, independently of user-field aggregates and sequence-group retractions. DELETE/UPDATE_BEFORE can leave a surviving row whose version must still be the latest effective KV version. ### Proposed behavior - Support sequence projections and predicates through Rust core and the existing Python read builder. - Follow Java's DataEvolutionSplitRead and ColumnarRowIterator: use the newest normal-file metadata provider; retain non-null physical versions and fill null versions from manifest metadata. - Keep metadata name resolution, scan pruning and residual filtering consistent, including mixed user/metadata predicates. - Follow Java aggregation/partial-update result metadata, including retractions and normalized result row kinds. - Remove the unused `scan.ignore-lost-files` option and require selected ROW sidecars to exist. The storage format and dependencies do not change. The paired Python change removes the sequence capability fallback and corrects pure-Python append metadata filtering and physical-version retention. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
