JingsongLi opened a new pull request, #1435:
URL: https://github.com/apache/paimon-rust/pull/1435

   ### Purpose
   
   Support Paimon/Spark SQL compatibility for maximum-partition lookup and 
discrete approximate percentiles in SQLContext.
   
   For example, `SELECT * FROM events WHERE dt = max_pt('events')` resolves the 
current maximum first-level partition during planning, and 
`approx_percentile(v, array(0.5, 0.4, 0.1), 100)` preserves Spark's percentile 
ordering and input type.
   
   ### Brief change log
   
   - Add `max_pt`, `sys.max_pt`, and catalog-qualified calls using the normal 
session table resolution, snapshot/branch options, and scan authorization. 
Compare partition values by their logical type and fold results into literals 
for partition pruning. Reject grants with row filters or column masks before 
reading raw partition values.
   - Add `approx_percentile` and its `percentile_approx` alias using Spark's 
mergeable Greenwald–Khanna summary, with constant accuracy and scalar/array 
percentages, discrete rank selection, typed results, DISTINCT, FILTER, and 
windows.
   - Validate percentile parameters before optimizer elimination. Cover decimal 
conversion and overflow, temporal/interval units, floating-point normalization, 
partial merges, and bounded summary memory.
   - Keep persistent views dynamic and reject metadata reads inside immutable 
catalog SQL functions. Reuse the canonical Java-compatible partition formatting 
helpers.
   
   ### Tests
   
   - `cargo +1.95.0 test --locked -p paimon-datafusion --all-targets` with the 
CI-required `RUST_MIN_STACK=16777216` — passed.
   - `cargo +1.95.0 test --locked -p paimon-datafusion --lib 
approx_percentile::quantile_summary` after the final Clippy repair — 6 passed.
   - `cargo +1.98.0 clippy --locked -p paimon-datafusion --all-targets 
--no-deps -- -D warnings` — passed.
   - `cargo +1.98.0 fmt --all -- --check` and `git diff --check` — passed.
   - Regression coverage includes restricted metadata grants through persistent 
views, negative submicrosecond intervals, Float16 DISTINCT signed zeros and NaN 
payloads, typed partition ordering, fresh snapshots, Spark percentile examples, 
parameter errors, partial aggregation, and moving windows.
   
   ### API and Format
   
   Adds SQL functions and public reexports of existing Java-compatible 
UTF-8/double formatting helpers. No table storage or catalog metadata format 
changes. Aggregate summary serialization is internal query state. Implicit SQL 
casts retain DataFusion semantics; decimal percentile arrays expose nullable 
elements to represent overflow safely.
   
   ### Documentation
   
   Updated `docs/src/sql.md` with signatures, examples, compatibility 
boundaries, authorization behavior, and resource characteristics.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to