[
https://issues.apache.org/jira/browse/SPARK-30752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-30752.
---------------------------------
Fix Version/s: 3.0.0
Resolution: Fixed
Issue resolved by pull request 27474
[https://github.com/apache/spark/pull/27474]
> Wrong result of to_utc_timestamp() on daylight saving day
> ---------------------------------------------------------
>
> Key: SPARK-30752
> URL: https://issues.apache.org/jira/browse/SPARK-30752
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.4.4, 3.0.0
> Reporter: Maxim Gekk
> Assignee: Maxim Gekk
> Priority: Major
> Fix For: 3.0.0
>
>
> The to_utc_timestamp() function returns wrong result when:
> * JVM system time zone is PST
> * the session local time zone is UTC
> * fromZone is Asia/Hong_Kong
> for the local date '2019-11-03T12:00:00', the result must be
> '2019-11-03T04:00:00'
> {code}
> scala> import java.util.TimeZone
> import java.util.TimeZone
> scala> import org.apache.spark.sql.catalyst.util.DateTimeUtils._
> import org.apache.spark.sql.catalyst.util.DateTimeUtils._
> scala> import org.apache.spark.sql.functions._
> import org.apache.spark.sql.functions._
> scala> TimeZone.setDefault(getTimeZone("PST"))
> scala> spark.conf.set("spark.sql.session.timeZone", "UTC")
> scala> val df = Seq("2019-11-03T12:00:00").toDF("localTs")
> df: org.apache.spark.sql.DataFrame = [localTs: string]
> scala> df.select(to_utc_timestamp(col("localTs"), "Asia/Hong_Kong")).show
> +-----------------------------------------+
> |to_utc_timestamp(localTs, Asia/Hong_Kong)|
> +-----------------------------------------+
> | 2019-11-03 03:00:00|
> +-----------------------------------------+
> {code}
>
> See
> https://www.worldtimebuddy.com/?qm=1&lid=8,1819729,100&h=8&date=2019-11-2&sln=21-22
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
