rank-p opened a new pull request, #27163: URL: https://github.com/apache/datafusion/pull/27163
## Which issue does this PR close? - Closes #27162. Related to lakehq/sail#2730. ## Rationale for this change Parquet sort declarations apply to individual row groups. Using the first group's declaration for the whole file can remove a required sort. For groups `[1, 4]` and `[2, 3]`, `ORDER BY a ASC NULLS FIRST LIMIT 2` returned `[1, 4]` instead of `[1, 2]`. ## What changes are included in this PR? Keep only the common leading sort keys whose row-group boundaries are ordered. Use typed bounds that account for NULLs, comparison order, and schema mapping. Missing or incompatible metadata leaves the required sort in place. ## What is the testing strategy for this PR? - Query regressions in `core/tests/parquet/ordering.rs` fail on the old code and pass with the fix, including `ORDER BY ... LIMIT`. - Query and metadata tests cover valid multi-group files, NULLs, multiple keys, missing metadata, projections, and floating-point order. - 301 Parquet crate tests, 13,535 extended Rust tests, and 546 SQL test files pass. Full-workspace Clippy passes with all features enabled. ## Are there any user-facing changes? Correct ordering and row selection when reading Parquet files whose row groups are sorted but the whole file is not. No public API changes. AI assistance: Codex. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
