andygrove opened a new issue, #4491:
URL: https://github.com/apache/datafusion-comet/issues/4491
## Describe the bug
`CometCast.isSupported` has no `MapType` arm; any cast involving a `MapType`
source or target falls into the default `case _ => unsupported(fromType,
toType)` and falls back to Spark.
However, the native side already has a `cast_map_to_map` function (see
`native/spark-expr/src/conversion_funcs/cast.rs` around line 423). The Scala
side just never routes to it, so `CAST(<map> AS MAP<...>)` falls back
unnecessarily.
Surfaced by the cast audit (collection PR queue).
## Expected behavior
Add a `(MapType, MapType)` arm to `CometCast.isSupported` that recursively
checks the source / target key and value types and returns `Compatible` (or
`Incompatible(reason)`) when both inner casts are supported. Wire the recursion
the same way `Array -> Array` and `Struct -> Struct` are handled in
`CometCast.scala`.
## Additional context
- Native impl:
`native/spark-expr/src/conversion_funcs/cast.rs::cast_map_to_map`
- Comet matrix: `CometCast.scala`
- For `Map -> String`, Spark formats as `{k1 -> v1, k2 -> v2}`; the native
path would need a `cast_map_to_string` analogous to the existing
`cast_struct_to_string`. Out of scope here; this issue is only the `Map -> Map`
case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
