MaxGekk commented on a change in pull request #28787:
URL: https://github.com/apache/spark/pull/28787#discussion_r438556136
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/RebaseDateTime.scala
##########
@@ -326,20 +326,34 @@ object RebaseDateTime {
*/
private[sql] def rebaseGregorianToJulianMicros(zoneId: ZoneId, micros:
Long): Long = {
val instant = microsToInstant(micros)
- var ldt = instant.atZone(zoneId).toLocalDateTime
+ val zonedDateTime = instant.atZone(zoneId)
+ var ldt = zonedDateTime.toLocalDateTime
if (ldt.isAfter(julianEndTs) && ldt.isBefore(gregorianStartTs)) {
ldt = LocalDateTime.of(gregorianStartDate, ldt.toLocalTime)
}
val cal = new Calendar.Builder()
- // `gregory` is a hybrid calendar that supports both
- // the Julian and Gregorian calendar systems
+ // `gregory` is a hybrid calendar that supports both the Julian and
Gregorian calendar systems
.setCalendarType("gregory")
.setDate(ldt.getYear, ldt.getMonthValue - 1, ldt.getDayOfMonth)
.setTimeOfDay(ldt.getHour, ldt.getMinute, ldt.getSecond)
- // Local time-line can overlaps, such as at an autumn daylight savings
cutover.
- // This setting selects the original local timestamp mapped to the given
`micros`.
- .set(Calendar.DST_OFFSET,
zoneId.getRules.getDaylightSavings(instant).toMillis.toInt)
.build()
+ // A local timestamp can have 2 instants in the cases of switching from:
+ // 1. Summer to winter time.
+ // 2. One standard time zone to another one. For example, Asia/Hong_Kong
switched from JST
+ // to HKT on 18 November, 1945 01:59:59 AM.
+ // Below we check that the original `instant` is earlier or later instant.
If it is an earlier
+ // instant, we take the standard and DST offsets of the previous day
otherwise of the next one.
+ val trans = zoneId.getRules.getTransition(ldt)
+ if (trans != null && trans.isOverlap) {
Review comment:
Only when overlapping happens, it shouldn't be so often. Usually, it
happens once per year. Please, see my comment in the code above.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
