weimingdiit opened a new issue, #2431: URL: https://github.com/apache/auron/issues/2431
**Is your feature request related to a problem? Please describe.** Auron Iceberg native changelog scan supports insert-only changelog tasks and materializes changelog metadata columns as per-task constant values. However, simple predicates on changelog metadata columns are not used to prune changelog scan tasks during native scan planning. For example, a query filtering by `_commit_snapshot_id` may still plan all insert-only changelog tasks in the selected changelog range. **Describe the solution you'd like** Use simple changelog metadata predicates to prune `AddedRowsScanTask`s before building native Iceberg scan tasks. The scope should be limited to task-level constants: - `_change_type` - `_commit_snapshot_id` - `_change_ordinal` Only simple predicate forms should be supported: - `=` - `IN` - `AND` Unsupported expressions such as `OR`, `NOT`, complex expressions, or mixed data-column predicates should not be used for task-level pruning. **Describe alternatives you've considered** One alternative is to rely only on the existing post-scan filter path. That keeps correctness, but it can require native scan to read changelog tasks that are known not to match based on task-level metadata. **Additional context** This change should not add support for delete, update, position delete, or equality delete changelog operations. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
