zhouyc-ali opened a new pull request, #471: URL: https://github.com/apache/paimon-cpp/pull/471
### Purpose Linked issue: N/A. Push supported predicates, projections and bitmap selections into native Vortex scans while preserving physical file row IDs and Paimon's residual-filter semantics. - Support comparisons, NULL checks, IN and NOT IN, with per-reader membership policies, deduplication and integer-range compression. Fall back when types, timestamp precision or NaN semantics cannot be preserved. - Reduce selective-read overhead by reusing default callback-reader sessions, lazily allocating dictionary filter caches and sharding file-local caches. Add explicit I/O coalescing modes while keeping the object-storage default. - Validate pinned native patch states and extend the benchmark runner to support Vortex. Cover projection/schema alignment, reader reconfiguration and callback lifetimes without changing the existing writer implementation. ### Tests - Full project build and CTest, Linux x86_64 / Clang 18 / Debug: default features plus Vortex, 33/33 targets; 7,402 tests passed, 22 skipped and 4 disabled. - Full project build with all optional CI features: 41/41 targets; 8,370 tests passed, 39 skipped and 4 disabled. External service/network tests were disabled. - Native Rust tests: 255 passed. Targeted Vortex C++ ASAN/UBSAN and unsuppressed LSAN: 86 passed. Core/Arrow/Rust dependencies were not fully sanitizer-instrumented. - Reader and predicate regressions include 512 deterministic differential native scans. Three SDK tests exercise 24 combinations covering residual filtering, reordered projection, late materialization, PK updates/deletes and schema evolution. - Repository-wide `pre-commit run --all-files` passed on all 5,473 tracked files with no modifications. Changed-file clang-tidy passed. ### API and Format No changes to public C++ API signatures, stored file formats or the table protocol. Add the reader options `vortex.read.membership-pushdown` and `vortex.read.io-coalesce`. The private Vortex FFI is extended; its patched header and native library must be rebuilt together. Readers continue to require SDK residual filtering (`EnablePredicateFilter(true)`) or equivalent filtering by the caller for exact results. ### Documentation Document the two Vortex reader options, their defaults and the residual-filter requirement in the existing reading guide (`docs/source/user_guide/read.rst`). ### Generative AI tooling Generated-by: OpenAI Codex 0.163.0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
