MaxGekk commented on issue #24195: [SPARK-25496][SQL] Deprecate from_utc_timestamp and to_utc_timestamp URL: https://github.com/apache/spark/pull/24195#issuecomment-476870628 I have traced what is happening in the example: 1. The `TIMESTAMP` typed literal takes `2011-12-25 09:00:00.123456`, and calls `stringToTimestamp` with `UTC-11:00` : https://github.com/apache/spark/blob/9cc925cda2c415deca8cf142a2a1774e81fda3fb/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala#L1637 which produces `1324843200123456`. This is _Sunday December 25, 2011 20:00:00 (pm) in time zone UTC_. 2. `FROM_UTC_TIMESTAMP` takes `1324843200123456` and `fromZone = GMT`, `toZone = Asia/Tokyo` (offset=32400000), and produces `1324875600123456` which is _Monday December 26, 2011 **05:00:00** (am) in time zone UTC (UTC)_ 3. `HOUR` invokes `DateTimeUtils.getHours` with `1324875600123456` and `timeZone = GMT` **(???)**, and produces just **`5`**. The `getHours` is called from https://github.com/apache/spark/blob/e6e9031d7b7458c9d88205d06cdcdd95a98b9537/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveInlineTables.scala#L103 It seems we have another issue in folding with wrong time zone here besides of `FROM_UTC_TIMESTAMP`.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
