hhr293 commented on code in PR #59394:
URL: https://github.com/apache/spark/pull/59394#discussion_r4268479883
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala:
##########
@@ -429,6 +431,21 @@ object Cast extends QueryErrorsBase {
case (fromField, toField) =>
needsTimeZone(fromField.dataType, toField.dataType)
}
+ // `castToString` recurses into ARRAY/MAP/STRUCT and formats each LTZ leaf
with `zoneId`, while
+ // the complex-to-complex cases above only match a target of the same
shape.
+ case (from, _: StringType) => from.existsRecursively(needsTimeZoneToString)
+ case _ => false
+ }
+
+ /**
+ * Whether rendering a value of this type as a string reads the session time
zone. Only the LTZ
+ * families carry an absolute instant that has to be localized;
TIMESTAMP_NTZ, DATE and TIME
+ * render on a fixed grid. VARIANT may hold a timestamp at runtime, as in
`needsTimeZone`.
+ */
+ private def needsTimeZoneToString(dt: DataType): Boolean = dt match {
+ case TimestampType | _: TimestampLTZNanosType | VariantType => true
Review Comment:
Thanks for the detailed review! You're right — nested VARIANT-to-STRING
rendering uses UTC and does not depend on the session time zone.
I'll remove VariantType from the helper, update the comment, and adjust the
tests to cover both nested and element-wise VARIANT casts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
