andygrove commented on code in PR #6320:
URL: https://github.com/apache/datafusion-comet/pull/6320#discussion_r4285487479
##########
docs/source/user-guide/latest/compatibility/expressions/_category_template/cast.md:
##########
@@ -161,6 +165,38 @@ Spark's behavior of using Java `BigDecimal.toString()`
semantics, which produces
notation (e.g. a value of 12300 stored as `Decimal(7,-2)` with unscaled value
123 is rendered
as `"1.23E+4"`).
+## Complex Types
+
+Comet casts arrays, structs, and maps by casting each element, field, key, and
value with the
+rules for primitive types, using the table for the same eval mode. A
complex-type cast is
+therefore compatible only when every cast it contains is compatible. For
example, casting
+`ARRAY<INT>` to `ARRAY<BIGINT>` has the same support level as casting `INT` to
`BIGINT`, and a
+struct cast with one unsupported field cast is unsupported. The rules apply at
every level of
+nesting, so casting `ARRAY<STRUCT<a: INT>>` to `ARRAY<STRUCT<a: BIGINT>>` also
has the same
+support level as casting `INT` to `BIGINT`.
+
+| From | To | Support level
|
+| ----------- | ------------- |
---------------------------------------------------------------------------- |
+| `ARRAY<T>` | `ARRAY<U>` | Same as `T` to `U`
|
+| `ARRAY<T>` | `STRING` | Same as `T` to `STRING`
|
+| `STRUCT` | `STRUCT` | Compatible if every field cast is compatible.
Fields are matched by position |
+| `STRUCT` | `STRING` | Compatible if the cast from every field to
`STRING` is compatible |
+| `MAP<K, V>` | `MAP<K2, V2>` | Compatible if `K` to `K2` and `V` to `V2` are
both compatible |
+| `MAP<K, V>` | `STRING` | Unsupported
|
+
+The following casts are exceptions to these rules:
+
+- `ARRAY<DATE>` can only be cast natively to `ARRAY<INT>` or `ARRAY<STRING>`.
Casts to other array
+ types are unsupported.
+- A cast from a `DATE` struct field or map value to a numeric or boolean type
is unsupported. In
+ Legacy mode these casts always return `NULL` (see [Date to Numeric
Types](#date-to-numeric-types)).
+- When `spark.sql.legacy.castComplexTypesToString.enabled=true`, casts from
arrays, structs, and
+ maps to `STRING` are unsupported, because Comet's native cast only produces
Spark's default
+ string format.
+
+In Try mode, a cast between maps fails with an error when a key cannot be
cast, where Spark returns
Review Comment:
Good point on the scope. I added a sentence saying the note only applies to
key casts that Spark accepts under TRY_CAST, such as narrowing BIGINT to INT,
and that Spark rejects a STRING key cast at analysis (39b23dc142). On ordering,
whichever of this PR and #6178 lands second should delete or update the note.
I'll check this page when #6178 merges, and I'll note the cast.md follow-up on
that PR so it isn't missed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
