Muneer320 opened a new pull request, #27151: URL: https://github.com/apache/datafusion/pull/27151
## Which issue does this PR close? Closes #27134. ## Rationale for this change For a table partitioned by `year` and `month`, a file at `year=2024/month=02.parquet` is incorrectly read as month `02.parquet`. The filename supplies a partition value even though the `month=.../` directory is missing. ## What changes are included in this PR? Require a following path component for each partition directory, so the final filename cannot supply a partition value. Preserve percent-decoding and the existing handling of incomplete partition paths. A filename containing `=` is still valid inside a complete partition directory structure. Extend the existing partial-partition regression and add SQL coverage alongside the related listing-table partition tests. Disabling the listing cache rebuilds session state and changes the default-catalog setting; reset that setting at the end for the runner's configuration check. ## What is the testing strategy for this PR? - On the initial source base, the unit regression fails before the fix; all 19 then-existing catalog-listing unit tests, 7 doc tests and `listing_table_partitions.slt` pass afterward. - Refreshed onto main at `888879d01c8131038ab8693cde227597ab5a434f`, retaining the new Hive NULL partition behavior and SQL coverage. The filename regression checks skipped incomplete paths, returned rows, partition filtering, and a `key=value` filename in a complete path. - On that refreshed base, `cargo fmt --all`, `cargo clippy --all-targets --all-features -- -D warnings`, and the repository's extended workspace test command all pass on a [GitHub-hosted runner](https://github.com/Muneer320/datafusion/actions/runs/42135548295), before committing this exact patch. That run includes all 21 current catalog-listing unit tests and the SQL logic-test harness. No existing microbenchmark calls this partition helper; this is a correctness fix and makes no performance claim. ## Are there any user-facing changes? Files whose filenames masquerade as missing partition directories are skipped, consistently with other incomplete partition paths. Complete partition paths retain their existing values. There is no public API change. AI assistance: Codex assisted with the implementation, regression tests and this description. The author has reviewed the current patch. The change follows the repository's existing parser and test patterns. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
