JingsongLi opened a new pull request, #1314:
URL: https://github.com/apache/paimon-rust/pull/1314

   ### Purpose
   
   Follow-up to #1236. Paimon scans report `UnknownPartitioning` even when 
table partition keys prove disjoint ranges, preventing DataFusion 55 from 
reusing that distribution for compatible aggregations and joins.
   
   ### Brief change log
   
   - Group whole splits using typed Arrow comparisons with `ASC NULLS LAST`, 
keeping equal keys together and combining adjacent key groups within 
`target_partitions`. Support Boolean, integer, string, date, millisecond time, 
Decimal128, and composite partition keys.
   - Use the shared scan construction path for regular, Variant, and audit-log 
scans, and expose range metadata in EXPLAIN. This declares distribution without 
claiming rows within each range are sorted.
   - Preserve original split assignments and `UnknownPartitioning` for 
unsupported types, unavailable partition keys, malformed metadata, 
authorization rules, and empty/single-partition/single-key scans.
   
   ### Tests
   
   Added 5 unit tests and 6 integration tests covering boundaries, duplicate 
keys and NULLs across files, composite keys and projection, 
unsupported/authorization fallbacks, aggregation exchange elimination, 
Hash/Sort-Merge Join alignment after PK updates, Variant pushdown, and 
audit-log scans.
   
   Passed:
   - `RUST_MIN_STACK=16777216 cargo test --locked -p paimon-datafusion --test 
range_partitioning`
   - `RUST_MIN_STACK=16777216 cargo test --locked -p paimon-datafusion 
--all-targets --no-fail-fast -- --skip test_create_tag_from_timestamp --skip 
test_rollback_to_timestamp`
   - `cargo clippy --locked -p paimon-datafusion --all-targets --no-deps -- -D 
warnings`
   - `cargo fmt --all -- --check`
   
   The unfiltered test run failed on `test_create_tag_from_timestamp` and 
`test_rollback_to_timestamp`; both failures also reproduce in a clean worktree 
at the unchanged base `5bb42a11`. Full dependency linting encounters six 
existing `clippy::nonminimal_bool` diagnostics in the core `paimon` crate.
   
   ### API and Format
   
   No public API or storage format changes. Eligible scans expose `Range` 
distribution metadata; scan parallelism can be lower than the target when there 
are fewer distinct partition keys.
   
   ### Documentation
   
   Updated `docs/src/sql.md` with supported types, grouping behavior, ordering 
guarantees, and fallback conditions.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to