JingsongLi opened a new pull request, #1314: URL: https://github.com/apache/paimon-rust/pull/1314
### Purpose Follow-up to #1236. Paimon scans report `UnknownPartitioning` even when table partition keys prove disjoint ranges, preventing DataFusion 55 from reusing that distribution for compatible aggregations and joins. ### Brief change log - Group whole splits using typed Arrow comparisons with `ASC NULLS LAST`, keeping equal keys together and combining adjacent key groups within `target_partitions`. Support Boolean, integer, string, date, millisecond time, Decimal128, and composite partition keys. - Use the shared scan construction path for regular, Variant, and audit-log scans, and expose range metadata in EXPLAIN. This declares distribution without claiming rows within each range are sorted. - Preserve original split assignments and `UnknownPartitioning` for unsupported types, unavailable partition keys, malformed metadata, authorization rules, and empty/single-partition/single-key scans. ### Tests Added 5 unit tests and 6 integration tests covering boundaries, duplicate keys and NULLs across files, composite keys and projection, unsupported/authorization fallbacks, aggregation exchange elimination, Hash/Sort-Merge Join alignment after PK updates, Variant pushdown, and audit-log scans. Passed: - `RUST_MIN_STACK=16777216 cargo test --locked -p paimon-datafusion --test range_partitioning` - `RUST_MIN_STACK=16777216 cargo test --locked -p paimon-datafusion --all-targets --no-fail-fast -- --skip test_create_tag_from_timestamp --skip test_rollback_to_timestamp` - `cargo clippy --locked -p paimon-datafusion --all-targets --no-deps -- -D warnings` - `cargo fmt --all -- --check` The unfiltered test run failed on `test_create_tag_from_timestamp` and `test_rollback_to_timestamp`; both failures also reproduce in a clean worktree at the unchanged base `5bb42a11`. Full dependency linting encounters six existing `clippy::nonminimal_bool` diagnostics in the core `paimon` crate. ### API and Format No public API or storage format changes. Eligible scans expose `Range` distribution metadata; scan parallelism can be lower than the target when there are fewer distinct partition keys. ### Documentation Updated `docs/src/sql.md` with supported types, grouping behavior, ordering guarantees, and fallback conditions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
