Piotr Skąpski created SPARK-31174:
-------------------------------------
Summary: unix_timestamp() function returning NULL values for
corner cases (daylight saving)
Key: SPARK-31174
URL: https://issues.apache.org/jira/browse/SPARK-31174
Project: Spark
Issue Type: Bug
Components: Spark Shell, SQL
Affects Versions: 2.3.4
Reporter: Piotr Skąpski
Running below code does not return 4 values due to one timestamp not being
correctly created (possible problem with daylight saving):
{code:java}
spark.sql("""SELECT from_unixtime(unix_timestamp('2020-03-08 01:00:00'),
'yyyyMMdd') t1, from_unixtime(unix_timestamp('2020-03-08 02:00:00'),
'yyyyMMdd') t2, from_unixtime(unix_timestamp('2020-03-08 03:00:00'),
'yyyyMMdd') t3, from_unixtime(unix_timestamp('2020-03-08 04:00:00'),
'yyyyMMdd') t4""").show
+--------+----+--------+--------+
| t1| t2| t3| t4|
+--------+----+--------+--------+
|20200308|null|20200308|20200308|
+--------+----+--------+--------+{code}
This unexpected NULL value caused us problems as we did not expect it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
