adriangb commented on code in PR #24125:
URL: https://github.com/apache/datafusion/pull/24125#discussion_r3815435027
##########
datafusion/physical-expr-adapter/src/schema_rewriter.rs:
##########
@@ -282,13 +311,130 @@ impl DefaultPhysicalExprAdapterRewriter {
return Ok(Transformed::yes(transformed));
}
+ if let Some(transformed) = self.try_narrow_struct_cast(&expr)? {
+ return Ok(Transformed::yes(transformed));
+ }
+
if let Some(column) = expr.downcast_ref::<Column>() {
return self.rewrite_column(Arc::clone(&expr), column);
}
Ok(Transformed::no(expr))
}
+ /// Rewrite `get_field(cast(s AS Struct<..>), 'f')` into
Review Comment:
It's a web of:
- This is a bug, the lasting fix that makes this a perf improvement and not
a correctness issue is to fall back to running filters after the projection
(FilterExec embedded in Parquet scan, can never fail).
- That work is blocked because of perf regressions, a lot of them related to
HashJoinExec dynamic filters (they go from never being evaluated anywhere to
being evaluated, and they are often regressions)
I might try to propose disabling the hash join dynamic filters by default.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
