tlfua commented on code in PR #7457:
URL: https://github.com/apache/datafusion-comet/pull/7457#discussion_r4489262953
##########
spark/src/main/scala/org/apache/comet/expressions/CometCast.scala:
##########
@@ -572,6 +572,32 @@ object CometCast
private[comet] def canProduceNullMapKey(cast: Cast): Boolean =
evalMode(cast) == CometEvalMode.TRY && mayNullMapKey(cast.child.dataType,
cast.dataType)
+ /**
+ * Whether `cast` cannot raise an error for any value. A TRY cast returns
NULL for a value it
+ * cannot convert, and so does a LEGACY cast between the atomic types listed
here. An ANSI cast
+ * cannot fail only where the conversion is always exact, such as a widening
one or a string to
+ * its bytes. A cast to or from a timestamp is excluded in every mode: the
native kernels go
+ * through chrono, which rejects years Spark accepts, and raise even for TRY.
+ */
+ private[comet] def cannotFail(cast: Cast): Boolean = {
+ val from = cast.child.dataType
+ val to = cast.dataType
+ def isTimestamp(t: DataType) = t == TimestampType || t == TimestampNTZType
+ !isTimestamp(from) && !isTimestamp(to) && (evalMode(cast) match {
+ case CometEvalMode.TRY => true
Review Comment:
Thanks, good catch. Fixed in 5a5b4230e. `CometCast.cannotFail` now looks
inside arrays, structs and maps: a timestamp anywhere in the source or target
type makes the cast able to fail, and so does a date in the source combined
with a string in the target. A nested cast applies the same chrono-based
kernels to each element, field, key and value, so they raise under TRY just
like the top-level cast (
https://github.com/apache/datafusion-comet/issues/7471 ).
`topk_later_key_ansi.sql` adds a `topk_nested` table with
`timestamp_micros(9000000000000000000)` and `date_from_unix_date(100000000)`,
and checks that these later keys fall back: `try_cast` of `array<timestamp>`
and `array<date>` to `STRING` and to `array<string>`, `struct<timestamp>` and
`struct<date>` to `STRING`, `struct<date>` to `struct<string>` and
`struct<timestamp>`, `struct<timestamp>` to `struct<date>`, `map<int, date>` to
`map<int, string>`, `map<timestamp, int>` to `map<string, int>`, and
`array<struct<timestamp>>` to `STRING`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
