Marek Byszewski created SPARK-34386:
---------------------------------------
Summary: "Proleptic" date off by 10 days when returned by
.collectAsList
Key: SPARK-34386
URL: https://issues.apache.org/jira/browse/SPARK-34386
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.0.1
Environment: Windows 10
Reporter: Marek Byszewski
Run the following commands using Spark 3.0.1:
{{scala> spark.sql("select to_timestamp('1582-10-05 02:12:34.997') as
data_console").show(false)}}
{{+-----------------------+}}
{{|data_console |}}
{{+-----------------------+}}
{{|*1582-10-05 02:12:34.997*|}}
{{+-----------------------+}}
{{scala> spark.sql("select to_timestamp('1582-10-05 02:12:34.997') as
data_console")}}
{{res3: org.apache.spark.sql.DataFrame = [data_console: timestamp]}}
{{scala> res3.collectAsList}}
{{res4: java.util.List[org.apache.spark.sql.Row] =
[[*1582-10-{color:#FF0000}15{color} 02:12:34.997*]]}}
Notice that the returned date is off by 10 days compared to the date returned
by the first command.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
