parthchandra commented on code in PR #2044:
URL: https://github.com/apache/datafusion-comet/pull/2044#discussion_r2274898989
##########
native/spark-expr/src/conversion_funcs/cast.rs:
##########
@@ -866,6 +867,40 @@ pub fn spark_cast(
}
}
+// copied from datafusion common scalar/mod.rs
+fn dict_from_values<K: ArrowDictionaryKeyType>(
Review Comment:
I looked into reusing the cast kernel. Looks like existing code already
would have fallen thru to the catch all branch which calls the cast kernel.
Even though `can_cast_types` appears to allow this, the cast actually fails
with
```
Caused by: org.apache.comet.CometNativeException: InternalError: Native cast
invoked for unsupported cast from Utf8 to Dictionary(Int32, Utf8).
```
The reason for the cast (which is implicit and does not show up in the plan)
is apparently because of a case statement like
```
CASE WHEN (_condition_) THEN _dict_encoded_column_ ELSE '' END
```
In such a case, we are implicitly trying to cast the array containing the
string value to a dict encoded array.
I'll open a follow up PR to make `dict_from_values` publich in DF and remove
the duplication
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
