Licht-T commented on code in PR #55610:
URL: https://github.com/apache/spark/pull/55610#discussion_r3193002831
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala:
##########
@@ -485,6 +485,43 @@ object DateTimeUtils extends SparkDateTimeUtils {
instantToMicros(truncated.toInstant)
}
+ /**
+ * Fast path for truncating to MINUTE/HOUR/DAY using offset arithmetic
instead of
+ * allocating a `ZonedDateTime` per row. The offset is resolved once for
`micros`; the
+ * truncation then runs as `floorMod` in local time. We fall back to
[[truncToUnit]] when
+ * the offset at the candidate truncated instant differs from the offset at
`micros`,
+ * which means the truncation crosses a DST/historical transition and the
local-time
+ * alignment we computed is no longer valid (see SPARK-30766/30857). The
check is
+ * skipped for fixed-offset zones. Sub-minute offsets (e.g.
America/Los_Angeles LMT
+ * -07:52:58, see SPARK-33404) and 30/45-minute offsets (Asia/Kolkata
+05:30, Asia/Kathmandu
+ * +05:45) are handled correctly by this path because the offset is applied
as part of
+ * the arithmetic; no offset-alignment guard is needed.
Review Comment:
Added the sentence into the Scaladoc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
