Maxim Gekk created SPARK-30752:
----------------------------------
Summary: Wrong result of to_utc_timestamp() on daylight saving day
Key: SPARK-30752
URL: https://issues.apache.org/jira/browse/SPARK-30752
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.4.4, 3.0.0
Reporter: Maxim Gekk
The to_utc_timestamp() function returns wrong result when:
* JVM system time zone is PST
* the session local time zone is UTC
* fromZone is Asia/Hong_Kong
for the local date '2019-11-03T12:00:00', the result must be
'2019-11-03T04:00:00'
{code}
scala> import java.util.TimeZone
import java.util.TimeZone
scala> import org.apache.spark.sql.catalyst.util.DateTimeUtils._
import org.apache.spark.sql.catalyst.util.DateTimeUtils._
scala> import org.apache.spark.sql.functions._
import org.apache.spark.sql.functions._
scala> TimeZone.setDefault(getTimeZone("PST"))
scala> spark.conf.set("spark.sql.session.timeZone", "UTC")
scala> val df = Seq("2019-11-03T12:00:00").toDF("localTs")
df: org.apache.spark.sql.DataFrame = [localTs: string]
scala> df.select(to_utc_timestamp(col("localTs"), "Asia/Hong_Kong")).show
+-----------------------------------------+
|to_utc_timestamp(localTs, Asia/Hong_Kong)|
+-----------------------------------------+
| 2019-11-03 03:00:00|
+-----------------------------------------+
{code}
See
https://www.worldtimebuddy.com/?qm=1&lid=8,1819729,100&h=8&date=2019-11-2&sln=21-22
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
