paleolimbot commented on issue #19550: URL: https://github.com/apache/datafusion/issues/19550#issuecomment-3720511216
> you had some ideas on how representation of field access might intersect with substrait My thought was that the high-level goal of improving nested data handling involves finding those expressions in logical and physical expressions to special case them or inject specialized behaviour in various ways. SQL and Substrait have special syntax for doing this (`a.b[2].c['my_map_key'].x` and https://substrait.io/expressions/field_references/, respectively) and a dedicated `Expr::NestedReference(Box<Expr>, NestedReference)` might help (or might not). DuckDB implements this at the logical as part of what would be our binary expression ( https://github.com/duckdb/duckdb/blob/2617272f9f899c173a2fab2548e3258de1067511/src/include/duckdb/parser/expression/operator_expression.hpp#L90-L121 ). You're correct that it's orthogonal to this issue...it would just change how you identify this type of expression (or maybe the optimizer rule that handles projection pushdown just checks for nested references until there is >1 logical operation for which there is a demonstrated advantage for pushing down in this way). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
