uros-b commented on code in PR #56375:
URL: https://github.com/apache/spark/pull/56375#discussion_r3409170576
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala:
##########
@@ -415,6 +429,16 @@ object Cast extends QueryErrorsBase {
// This conversion stays explicit-only.
case (_: TimestampNTZNanosType, TimestampNTZType) => false
case (_: TimestampLTZNanosType, TimestampType) => false
+ // SPARK-57323: DATE <-> nanosecond-precision timestamp requires an
explicit CAST in both
Review Comment:
The nanos -> DATE direction blocking store assignment is clearly right
(lossy — drops time-of-day + sub-micros, consistent with the SPARK-57293 nanos
-> micros rule).
The one I'd like to confirm is DATE -> nanos. It's lossless (midnight, zero
fraction), and micro DATE -> TIMESTAMP[_NTZ] is allowed as a store assignment
via the (_: DatetimeType, _: DatetimeType) => true catch-all. This PR makes
nanos stricter than micro for the same conceptual cast, so e.g. `INSERT INTO
t_nanos(ts9) SELECT date_col` fails (whereas the micro equivalent succeeds).
The rationale in the comment (stay strict while the types are unreleased;
loosening later is compatible, tightening is not) is reasonable, but I want to
make sure we're deliberately accepting the micro/nanos divergence rather than
just inheriting it. Do we want DATE -> nanos to remain explicit-only, or be a
store assignment like micro?
Minor, separate note: forceNullable marks nanos NTZ -> DATE non-nullable,
which is more precise than micro TIMESTAMP_NTZ -> DATE (currently forced
nullable via the catch-all). That's an improvement, just flagging the
intentional inconsistency.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
