alamb opened a new issue, #10181: URL: https://github.com/apache/datafusion/issues/10181
### Is your feature request related to a problem or challenge? In 37.0.0 many of the built in functions have been migrated to `UDF`s as described on #8045 . The migration is completed in 38.0.0 One part of this change is that now certain `Expr`s must be rewritten into the appropriate functions. Most notably `get_field` that extracts a field from a `Struct` The rewrite happens automatically as part of the logical planner (in the [Analyzer pass](https://github.com/apache/arrow-datafusion/blob/060e67e8ff3b57ab695daeb28cf7175c4e2c568c/datafusion/functions-array/src/rewrite.rs#L151-L158)) However if you bypass those passes it will not happen Yeah you need to use the FunctionRewrtiter here (with the relevant rewriter registered) https://github.com/apache/arrow-datafusion/blob/0573f78c7e7a4d94c3204cee464b3860479e0afb/datafusion/optimizer/src/analyzer/function_rewrite.rs#L33 ## Example An example from discord: [link](https://discord.com/channels/885562378132000778/1166447479609376850/1229122082256851054) is: ```rust let schema = Schema::new(vec![ Field::new("id", DataType::Utf8, true), Field::new( "props", DataType::Struct(Fields::from(vec![Field::new("a", DataType::Utf8, true)])), true, ), ]); println!("schema {:?}", schema); let df_schema = DFSchema::try_from(schema.clone()).unwrap(); let plan = table_scan(Some("props_test"), &schema, None)? .filter(col("props").field("a").eq(lit("2021-02-02")))? .build()?; println!("logical plan {:?}", plan); let phys = DefaultPhysicalPlanner::default().create_physical_expr(&plan.expressions()[0], &df_schema, &SessionContext::new().state())?; println!("phys {:?}", phys); Ok(()) ``` This returns an error "NamedStructField should be rewritten in OperatorToFunction" ### Describe the solution you'd like _No response_ ### Describe alternatives you've considered One potential workaround is to call [`get_field`](https://docs.rs/datafusion/latest/datafusion/functions/core/expr_fn/fn.get_field.html) directly rather than `Expr::field` So instead of ```rust let plan = table_scan(Some("props_test"), &schema, None)? .filter(col("props").field("a").eq(lit("2021-02-02")))? .build()?; ``` call like ```rust let plan = table_scan(Some("props_test"), &schema, None)? .filter(get_field(col("props", "a")).eq(lit("2021-02-02")))? .build()?; ``` ### Additional context @ion-elgreco is seeing the same issue in Delta-rs: https://github.com/apache/datafusion/issues/9904#issuecomment-2069359171 > I tried it with 37.1.0 in delta-rs, but we still get this error: internal error: entered unreachable code: NamedStructField should be rewritten in OperatorToFunction, wasn't this regression fixed? @westonpace brings it up in discord [link](https://discord.com/channels/885562378132000778/1166447479609376850/1232040636350468197) Another report in discord: [link](https://discord.com/channels/885562378132000778/1166447479609376850/1229122082256851054) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
