Maxim Gekk created SPARK-30766:
----------------------------------
Summary: Wrong truncation of old timestamps to hours and days
Key: SPARK-30766
URL: https://issues.apache.org/jira/browse/SPARK-30766
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.0.0
Reporter: Maxim Gekk
The date_trunc() function incorrectly truncates old timestamps to HOUR and DAY:
{code:scala}
scala> import org.apache.spark.sql.functions._
import org.apache.spark.sql.functions._
scala> :paste
// Entering paste mode (ctrl-D to finish)
Seq("0010-01-01 01:02:03.123456").toDF()
.select($"value".cast("timestamp").as("ts"))
.select(date_trunc("HOUR", $"ts").cast("string"))
.show(false)
// Exiting paste mode, now interpreting.
+------------------------------------+
|CAST(date_trunc(HOUR, ts) AS STRING)|
+------------------------------------+
|0010-01-01 01:30:17 |
+------------------------------------+
{code}
the result must be *0010-01-01 01:00:00*
{code:scala}
scala> :paste
// Entering paste mode (ctrl-D to finish)
Seq("0010-01-01 01:02:03.123456").toDF()
.select($"value".cast("timestamp").as("ts"))
.select(date_trunc("DAY", $"ts").cast("string"))
.show(false)
// Exiting paste mode, now interpreting.
+-----------------------------------+
|CAST(date_trunc(DAY, ts) AS STRING)|
+-----------------------------------+
|0010-01-01 23:30:17 |
+-----------------------------------+
{code}
the result must be *0010-01-01 00:00:00*
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
