adriangb commented on code in PR #16086:
URL: https://github.com/apache/datafusion/pull/16086#discussion_r2100900247
##########
datafusion/datasource-parquet/src/opener.rs:
##########
@@ -178,7 +182,7 @@ impl FileOpener for ParquetOpener {
// Build predicates for this specific file
let (pruning_predicate, page_pruning_predicate) =
build_pruning_predicates(
predicate.as_ref(),
- &physical_file_schema,
+ &logical_file_schema,
Review Comment:
I would like it to operate at the physical layer and maybe we can get back
there through future work.
My immediate goal merging this PR was to fix the pretty bad regression I had
introduced. A step backwards to avoid running into a wall.
But I also am not sure that we can even make it operate at the physical
layer properly: there's a lot of logic that goes into determining what casts to
do and such that happens at the logical layer. We'd have to re-create all of
that at the physical layer because if the types of the columns change
everything has to be re-evaluated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
