[
https://issues.apache.org/jira/browse/SPARK-40791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17619663#comment-17619663
]
Sean R. Owen commented on SPARK-40791:
--------------------------------------
Well it seems like Java has changed its mind or had a fix, that happens to
coincide again with earlier Spark behavior. Hm. I think for the moment we need
to stick to the current behavior so that Spark's behavior is consistent, and,
such that it matches currently supported Java versions.
Does Spark work as advertised in Java 19 or is there now a bug / behavior
change in Spark 3 on Java 19?
> The semantics of `F` in `DateTimeFormatter` have changed
> --------------------------------------------------------
>
> Key: SPARK-40791
> URL: https://issues.apache.org/jira/browse/SPARK-40791
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 3.4.0
> Reporter: Yang Jie
> Priority: Major
>
> {code:java}
> val createSql =
> """
> |create temporary view v as select col from values
> | (timestamp '1582-06-01 11:33:33.123UTC+080000'),
> | (timestamp '1970-01-01 00:00:00.000Europe/Paris'),
> | (timestamp '1970-12-31 23:59:59.999Asia/Srednekolymsk'),
> | (timestamp '1996-04-01 00:33:33.123Australia/Darwin'),
> | (timestamp '2018-11-17 13:33:33.123Z'),
> | (timestamp '2020-01-01 01:33:33.123Asia/Shanghai'),
> | (timestamp '2100-01-01 01:33:33.123America/Los_Angeles') t(col)
> | """.stripMargin
> sql(createSql)
> withSQLConf(SQLConf.WHOLESTAGE_CODEGEN_ENABLED.key -> false.toString) {
> val rows = sql("select col, date_format(col, 'F') from v").collect()
> // scalastyle:off
> rows.foreach(println)
> } {code}
>
> Before Java 19, the result is
>
> {code:java}
> [1582-05-31 19:40:35.123,3]
> [1969-12-31 15:00:00.0,3]
> [1970-12-31 04:59:59.999,3]
> [1996-03-31 07:03:33.123,3]
> [2018-11-17 05:33:33.123,3]
> [2019-12-31 09:33:33.123,3]
> [2100-01-01 01:33:33.123,1] {code}
> Java 19
>
> {code:java}
> [1582-05-31 19:40:35.123,5]
> [1969-12-31 15:00:00.0,5]
> [1970-12-31 04:59:59.999,5]
> [1996-03-31 07:03:33.123,5]
> [2018-11-17 05:33:33.123,3]
> [2019-12-31 09:33:33.123,5]
> [2100-01-01 01:33:33.123,1] {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
