adriangb commented on code in PR #24125:
URL: https://github.com/apache/datafusion/pull/24125#discussion_r3749148601
##########
datafusion/physical-expr-adapter/src/schema_rewriter.rs:
##########
@@ -282,13 +311,130 @@ impl DefaultPhysicalExprAdapterRewriter {
return Ok(Transformed::yes(transformed));
}
+ if let Some(transformed) = self.try_narrow_struct_cast(&expr)? {
+ return Ok(Transformed::yes(transformed));
+ }
+
if let Some(column) = expr.downcast_ref::<Column>() {
return self.rewrite_column(Arc::clone(&expr), column);
}
Ok(Transformed::no(expr))
}
+ /// Rewrite `get_field(cast(s AS Struct<..>), 'f')` into
Review Comment:
Both of your intuitions is right. The major source of fiction right now is
that we have to decide to accept filters before seeing the physical schema, but
then apply them against the physical schema. Thus situations like this bug
arise where a cast must be introduced but that changes our ability to evaluate
the filter or not. Today that causes a correctness bug. The solution to turn
this from a correctness problem into a performance optimization problem is
https://github.com/apache/datafusion/pull/22384.
That said I'll look into the suggestion and see if there's an alternative
implementation for this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
