vbarua commented on code in PR #15854:
URL: https://github.com/apache/datafusion/pull/15854#discussion_r2067003785
##########
datafusion/substrait/src/logical_plan/producer.rs:
##########
@@ -1590,6 +1590,21 @@ pub fn from_cast(
schema: &DFSchemaRef,
) -> Result<Expression> {
let Cast { expr, data_type } = cast;
+ // since substrait Null must be typed, so if we see a cast(null, dt), we
make it a typed null
+ if let Expr::Literal(lit) = expr.as_ref() {
+ if lit.is_null() {
Review Comment:
The one thing I'm not sure about is the usage of `is_null()` here. I think
the logic inside that works for non-compound types here:
```rust
pub fn is_null(&self) -> bool {
match self {
ScalarValue::Boolean(v) => v.is_none(),
ScalarValue::Null => true,
ScalarValue::Float16(v) => v.is_none(),
```
But for non-compound types I'm not sure if it does the right thing for the
fold you're introducing:
```rust
ScalarValue::List(arr) => arr.len() == arr.null_count(),
ScalarValue::LargeList(arr) => arr.len() == arr.null_count(),
ScalarValue::FixedSizeList(arr) => arr.len() == arr.null_count(),
ScalarValue::Struct(arr) => arr.len() == arr.null_count(),
ScalarValue::Map(arr) => arr.len() == arr.null_count(),
```
because I don't think that it make sense to flatten empty compound types. If
I understand how this would work, something like
```
Cast(List[], List<Int32>)
```
would get flattened into a null literal of type `List<i32>` which is subtly
wrong, because it should be a empty `List<i32>`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
