adriangb commented on code in PR #18136:
URL: https://github.com/apache/datafusion/pull/18136#discussion_r2616094441
##########
datafusion/proto/src/logical_plan/from_proto.rs:
##########
@@ -528,7 +528,8 @@ pub fn parse_expr(
codec,
)?);
let data_type = cast.arrow_type.as_ref().required("arrow_type")?;
- Ok(Expr::Cast(Cast::new(expr, data_type)))
+ let field = Field::new("", data_type,
cast.nullable.unwrap_or(true));
Review Comment:
It feels like there should be an `UnnamedField` or
`DataTypeWithNullabilityAndMetadata`. I saw similar patterns in the literal
expressions.
##########
datafusion/expr/src/expr_schema.rs:
##########
@@ -630,9 +631,18 @@ impl ExprSchemable for Expr {
func.return_field_from_args(args)
}
// _ => Ok((self.get_type(schema)?, self.nullable(schema)?)),
- Expr::Cast(Cast { expr, data_type }) => expr
+ Expr::Cast(Cast { expr, field }) => expr
.to_field(schema)
- .map(|(_, f)| f.retyped(data_type.clone())),
+ .map(|(_, f)| {
+ // This currently propagates the nullability of the input
+ // expression as the resulting physical expression does
+ // not currently consider the nullability specified here
+ f.as_ref()
+ .clone()
+ .with_data_type(field.data_type().clone())
+ .with_metadata(f.metadata().clone())
Review Comment:
I didn't fully understand this comment / what's going on here. I'm sure it's
my fault but could you try rewording the comment, maybe giving `f` a more
meaningful name and replacing `_` with
`_unused_variable_that_has_a_useful_name`?
##########
datafusion/expr/src/expr.rs:
##########
@@ -812,13 +821,20 @@ pub struct TryCast {
/// The expression being cast
pub expr: Box<Expr>,
/// The `DataType` the expression will yield
- pub data_type: DataType,
+ pub field: FieldRef,
Review Comment:
Should we keep `data_type` as a deprecated field that we populate from
`field.data_type()` for a couple of releases?
##########
datafusion/physical-expr/src/planner.rs:
##########
@@ -287,16 +287,44 @@ pub fn create_physical_expr(
};
Ok(expressions::case(expr, when_then_expr, else_expr)?)
}
- Expr::Cast(Cast { expr, data_type }) => expressions::cast(
- create_physical_expr(expr, input_dfschema, execution_props)?,
- input_schema,
- data_type.clone(),
- ),
- Expr::TryCast(TryCast { expr, data_type }) => expressions::try_cast(
- create_physical_expr(expr, input_dfschema, execution_props)?,
- input_schema,
- data_type.clone(),
- ),
+ Expr::Cast(Cast { expr, field }) => {
+ if !field.metadata().is_empty() {
+ let (_, src_field) = expr.to_field(input_dfschema)?;
+ return plan_err!(
+ "Cast from {} to {} is not supported",
+ format_type_and_metadata(
+ src_field.data_type(),
+ Some(src_field.metadata()),
+ ),
+ format_type_and_metadata(field.data_type(),
Some(field.metadata()))
+ );
+ }
+
+ expressions::cast(
+ create_physical_expr(expr, input_dfschema, execution_props)?,
+ input_schema,
+ field.data_type().clone(),
Review Comment:
This does end up tying into #19097: I think they'd work well together, we'd
just want to pass the field directly here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
