friendlymatthew commented on code in PR #20913:
URL: https://github.com/apache/datafusion/pull/20913#discussion_r2931762543
##########
datafusion/datasource-parquet/src/row_filter.rs:
##########
@@ -448,10 +442,39 @@ impl TreeNodeVisitor<'_> for PushdownChecker<'_> {
.and_then(|a| a.as_any().downcast_ref::<Column>())
{
let return_type = func.return_type();
- if !DataType::is_nested(return_type) {
- if let Some(recursion) =
self.check_struct_field_column(column.name())
- {
- return Ok(recursion);
+
+ if !DataType::is_nested(return_type)
+ || self.is_nested_type_supported(return_type)
+ {
+ // try to resolve all field name arguments to strinrg
literals
+ // if any argument is not a string literal, we can not
determine the exact
Review Comment:
In practice, `get_field` will always receive string literals. This is
enforced at different levels:
- At runtime, the udf [requires all field_name args to be
scalars](https://github.com/apache/datafusion/blob/d2278a90b4543939cefb0f3ffbea8b025fe922f0/datafusion/functions/src/core/getfield.rs#L399-L408)
- At planning time, field names [must be non-empty
strings](https://github.com/apache/datafusion/blob/d2278a90b4543939cefb0f3ffbea8b025fe922f0/datafusion/functions/src/core/getfield.rs#L342-L351)
The `None` fallback is purely defensive. It's not likely that we'll hit this
in practice, but in the case we do, we'll just read out the entire struct.
Though, we can also err here like we did before
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
