MaxGekk commented on issue #24195: [SPARK-25496][SQL] Deprecate from_utc_timestamp and to_utc_timestamp URL: https://github.com/apache/spark/pull/24195#issuecomment-476812283 @dilipbiswal Let's look at the first example, and assume that we are in the `UTC-11:00` time zone: ```scala scala> spark.conf.set("spark.sql.session.timeZone", "UTC-11:00") ``` and apply a function which assumes its input of `TimestampType` like `HOUR`: ```scala scala> spark.sql("values HOUR(FROM_UTC_TIMESTAMP(TIMESTAMP '2011-12-25 09:00:00.123456', 'Asia/Tokyo'))").show(false) +----+ |col1| +----+ |5 | +----+ ``` The name of the function tells us we converted from a timestamp from UTC, so, in UTC the hour is `09` but we got `5` which does match to the time zone offset of `11` hours. In general, problems occur when you apply functions or expressions to output of `FROM_UTC_TIMESTAMP`/`TO_UTC_TIMESTAMP` because such functions expect microseconds since epoch in UTC time zone but get as an input something different due `FROM_UTC_TIMESTAMP`/`TO_UTC_TIMESTAMP` shift UTC time points.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
