friendlymatthew commented on issue #8881: URL: https://github.com/apache/arrow-rs/issues/8881#issuecomment-3672128084
> > I'd like to be able to coerce union data types in datafusion. At work, we interact with a lot of JSON and accessing data with -> returns Union types. And being able to apply a filter like json_val->'id' = 123 would be nice > > In DataFusion, I would expect the type coercion logic to handle this > > Specifically, if the type of `json_val->'id'` is `DataType::Union` and the type of `123` is `Int32`, I would expect the datafusion comparsion coercion logic to add a cast so the resulting expression is like > > (json_val->'id')::int = 123 > That is the https://docs.rs/datafusion/latest/datafusion/logical_expr/type_coercion/binary/fn.comparison_coercion.html function would handle adding the coercion Hi @alamb, I have a PR open on pydantic's datafusion fork that adds the necessary coercion logic: https://github.com/pydantic/datafusion/pull/49 It seems to work well in our system! I'd be curious to get your thoughts -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
