zjw1111 opened a new pull request, #553: URL: https://github.com/apache/paimon-cpp/pull/553
### Purpose Continue the Vortex predicate pushdown and selective-read work started by @zhouyc-ali. [Original pull request #471](https://github.com/apache/paimon-cpp/pull/471) Related issue: #319. - Push supported predicates, projections, and bitmap selections into Vortex scans while preserving physical file row IDs. Fall back to candidate rows when a predicate cannot be pushed safely. - Make membership limits and I/O coalescing configurable per reader. Bound Arrow batch materialization before export, including nested values. - Normalize timezone metadata for newly written TIMESTAMP_LTZ columns to UTC while preserving their epoch values. Add ASF headers to the dependency patches and include them in RAT checks. ### Tests - Vortex predicate converter tests cover comparisons, null checks, IN and NOT IN, nested AND/OR expressions, and safe fallback for unsupported values and oversized lists. - Vortex file reader tests cover sparse and dense selections, physical row IDs, projection and reader reconfiguration, user columns named `_row_idx`, and callback I/O and Arrow batch ownership. - Rust FFI tests cover bounded export of nested arrays, date and timestamp scalar bounds, I/O coalescing, and callback error and cancellation handling. - Vortex table-read integration tests cover late materialization with native filtering and the complete candidate and exact results when an oversized NOT IN list falls back. ### API and Format Vortex readers accept `vortex.read.in-pushdown.max-values`, `vortex.read.io-coalesce.max-gap`, and `vortex.read.io-coalesce.max-size`. The private Vortex FFI adds scan and scalar capabilities. New TIMESTAMP_LTZ files record UTC timezone metadata and preserve the stored epoch values. ### Documentation Document the Vortex reader options and defaults, predicate fallback rules, residual filtering, and TIMESTAMP_LTZ storage in `docs/source/api/file_format.rst`. ### Generative AI tooling Generated-by: OpenAI Codex 0.167.0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
