vbarua commented on code in PR #15854:
URL: https://github.com/apache/datafusion/pull/15854#discussion_r2067003785
##########
datafusion/substrait/src/logical_plan/producer.rs:
##########
@@ -1590,6 +1590,21 @@ pub fn from_cast(
schema: &DFSchemaRef,
) -> Result<Expression> {
let Cast { expr, data_type } = cast;
+ // since substrait Null must be typed, so if we see a cast(null, dt), we
make it a typed null
+ if let Expr::Literal(lit) = expr.as_ref() {
+ if lit.is_null() {
Review Comment:
The one thing I'm not sure about is the usage of `is_null()` here. I think
the logic inside that works for non-compound types here:
```rust
pub fn is_null(&self) -> bool {
match self {
ScalarValue::Boolean(v) => v.is_none(),
ScalarValue::Null => true,
ScalarValue::Float16(v) => v.is_none(),
```
But for compound types I'm not sure if it does the right thing for the fold
you're introducing:
```rust
ScalarValue::List(arr) => arr.len() == arr.null_count(),
ScalarValue::LargeList(arr) => arr.len() == arr.null_count(),
ScalarValue::FixedSizeList(arr) => arr.len() == arr.null_count(),
ScalarValue::Struct(arr) => arr.len() == arr.null_count(),
ScalarValue::Map(arr) => arr.len() == arr.null_count(),
```
because I don't think that it make sense to flatten empty compound types. If
I understand how this would work, something like
```
Cast(List[], List<Int32>)
```
would get flattened into a null literal of type `List<i32>` which is subtly
wrong, because it should be an empty `List<i32>`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
